Aug 04 10:27:40.266329 ip-10-0-130-250 systemd[1]: Starting Kubernetes Kubelet... Aug 04 10:27:40.638019 ip-10-0-130-250 kubenswrapper[2535]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 10:27:40.638019 ip-10-0-130-250 kubenswrapper[2535]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Aug 04 10:27:40.638019 ip-10-0-130-250 kubenswrapper[2535]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 10:27:40.638019 ip-10-0-130-250 kubenswrapper[2535]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Aug 04 10:27:40.638019 ip-10-0-130-250 kubenswrapper[2535]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 10:27:40.639461 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.639349 2535 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Aug 04 10:27:40.647036 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647015 2535 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 10:27:40.647036 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647032 2535 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 10:27:40.647036 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647035 2535 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 10:27:40.647036 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647038 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 10:27:40.647036 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647041 2535 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 10:27:40.647036 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647044 2535 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 10:27:40.647258 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647048 2535 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 10:27:40.647258 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647052 2535 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 10:27:40.647258 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647054 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 10:27:40.647258 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647057 2535 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 10:27:40.647258 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647060 2535 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 10:27:40.647258 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647063 2535 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 10:27:40.647258 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647083 2535 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 10:27:40.647258 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647086 2535 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 10:27:40.647258 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647090 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 10:27:40.647258 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647092 2535 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 10:27:40.647258 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647095 2535 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 10:27:40.647258 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647100 2535 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 10:27:40.647258 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647103 2535 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 10:27:40.647258 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647106 2535 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 10:27:40.647258 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647109 2535 feature_gate.go:328] unrecognized feature gate: Example Aug 04 10:27:40.647258 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647112 2535 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 10:27:40.647258 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647115 2535 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 10:27:40.647258 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647117 2535 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 10:27:40.647258 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647120 2535 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 10:27:40.647258 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647123 2535 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 10:27:40.647727 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647125 2535 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 10:27:40.647727 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647129 2535 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 10:27:40.647727 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647132 2535 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 10:27:40.647727 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647134 2535 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 10:27:40.647727 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647137 2535 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 10:27:40.647727 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647140 2535 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 10:27:40.647727 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647146 2535 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 10:27:40.647727 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647149 2535 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 10:27:40.647727 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647151 2535 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 10:27:40.647727 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647154 2535 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 10:27:40.647727 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647156 2535 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 10:27:40.647727 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647159 2535 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 10:27:40.647727 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647162 2535 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 10:27:40.647727 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647165 2535 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 10:27:40.647727 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647168 2535 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 10:27:40.647727 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647171 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 10:27:40.647727 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647173 2535 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 10:27:40.647727 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647176 2535 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 10:27:40.647727 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647179 2535 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 10:27:40.647727 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647181 2535 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 10:27:40.648244 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647184 2535 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 10:27:40.648244 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647186 2535 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 10:27:40.648244 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647190 2535 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 10:27:40.648244 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647192 2535 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 10:27:40.648244 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647195 2535 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 10:27:40.648244 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647200 2535 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 10:27:40.648244 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647204 2535 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 10:27:40.648244 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647207 2535 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 10:27:40.648244 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647210 2535 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 10:27:40.648244 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647213 2535 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 10:27:40.648244 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647216 2535 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 10:27:40.648244 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647219 2535 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 10:27:40.648244 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647222 2535 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 10:27:40.648244 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647225 2535 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 10:27:40.648244 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647228 2535 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 10:27:40.648244 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647231 2535 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 10:27:40.648244 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647234 2535 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 10:27:40.648244 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647237 2535 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 10:27:40.648675 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647239 2535 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 10:27:40.648675 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647242 2535 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 10:27:40.648675 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647244 2535 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 10:27:40.648675 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647247 2535 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 10:27:40.648675 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647250 2535 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 10:27:40.648675 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647252 2535 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 10:27:40.648675 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647255 2535 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 10:27:40.648675 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647258 2535 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 10:27:40.648675 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647261 2535 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 10:27:40.648675 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647263 2535 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 10:27:40.648675 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647267 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 10:27:40.648675 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647269 2535 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 10:27:40.648675 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647272 2535 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 10:27:40.648675 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647274 2535 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 10:27:40.648675 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647277 2535 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 10:27:40.648675 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647280 2535 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 10:27:40.648675 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647283 2535 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 10:27:40.648675 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647285 2535 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 10:27:40.648675 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647288 2535 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 10:27:40.648675 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647291 2535 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 10:27:40.649218 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647294 2535 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 10:27:40.649218 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647296 2535 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 10:27:40.649218 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647299 2535 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 10:27:40.649218 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647301 2535 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 10:27:40.649218 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647304 2535 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 10:27:40.649218 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647307 2535 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 10:27:40.649218 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647310 2535 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 10:27:40.649218 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647312 2535 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 10:27:40.649218 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647315 2535 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 10:27:40.649218 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647317 2535 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 10:27:40.649218 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647320 2535 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 10:27:40.649218 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647323 2535 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 10:27:40.649218 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647325 2535 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 10:27:40.649218 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647328 2535 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 10:27:40.649218 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.647330 2535 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 10:27:40.649218 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648017 2535 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 10:27:40.649218 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648024 2535 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 10:27:40.649218 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648027 2535 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 10:27:40.649218 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648029 2535 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 10:27:40.649218 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648034 2535 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 10:27:40.649218 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648036 2535 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 10:27:40.649712 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648039 2535 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 10:27:40.649712 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648042 2535 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 10:27:40.649712 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648045 2535 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 10:27:40.649712 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648047 2535 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 10:27:40.649712 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648050 2535 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 10:27:40.649712 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648053 2535 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 10:27:40.649712 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648055 2535 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 10:27:40.649712 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648058 2535 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 10:27:40.649712 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648061 2535 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 10:27:40.649712 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648063 2535 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 10:27:40.649712 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648081 2535 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 10:27:40.649712 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648087 2535 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 10:27:40.649712 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648090 2535 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 10:27:40.649712 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648092 2535 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 10:27:40.649712 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648095 2535 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 10:27:40.649712 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648100 2535 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 10:27:40.649712 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648104 2535 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 10:27:40.649712 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648107 2535 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 10:27:40.649712 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648110 2535 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 10:27:40.650203 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648113 2535 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 10:27:40.650203 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648115 2535 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 10:27:40.650203 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648118 2535 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 10:27:40.650203 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648121 2535 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 10:27:40.650203 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648124 2535 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 10:27:40.650203 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648127 2535 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 10:27:40.650203 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648136 2535 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 10:27:40.650203 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648139 2535 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 10:27:40.650203 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648142 2535 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 10:27:40.650203 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648144 2535 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 10:27:40.650203 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648147 2535 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 10:27:40.650203 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648151 2535 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 10:27:40.650203 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648154 2535 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 10:27:40.650203 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648157 2535 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 10:27:40.650203 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648159 2535 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 10:27:40.650203 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648163 2535 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 10:27:40.650203 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648166 2535 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 10:27:40.650203 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648168 2535 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 10:27:40.650203 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648171 2535 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 10:27:40.650203 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648174 2535 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 10:27:40.650698 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648177 2535 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 10:27:40.650698 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648180 2535 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 10:27:40.650698 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648183 2535 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 10:27:40.650698 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648186 2535 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 10:27:40.650698 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648188 2535 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 10:27:40.650698 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648191 2535 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 10:27:40.650698 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648194 2535 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 10:27:40.650698 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648197 2535 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 10:27:40.650698 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648199 2535 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 10:27:40.650698 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648202 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 10:27:40.650698 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648204 2535 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 10:27:40.650698 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648207 2535 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 10:27:40.650698 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648209 2535 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 10:27:40.650698 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648212 2535 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 10:27:40.650698 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648214 2535 feature_gate.go:328] unrecognized feature gate: Example Aug 04 10:27:40.650698 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648217 2535 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 10:27:40.650698 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648220 2535 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 10:27:40.650698 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648222 2535 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 10:27:40.650698 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648225 2535 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 10:27:40.650698 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648227 2535 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 10:27:40.651189 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648230 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 10:27:40.651189 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648233 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 10:27:40.651189 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648235 2535 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 10:27:40.651189 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648238 2535 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 10:27:40.651189 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648241 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 10:27:40.651189 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648243 2535 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 10:27:40.651189 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648246 2535 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 10:27:40.651189 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648249 2535 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 10:27:40.651189 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648252 2535 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 10:27:40.651189 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648255 2535 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 10:27:40.651189 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648258 2535 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 10:27:40.651189 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648260 2535 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 10:27:40.651189 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648263 2535 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 10:27:40.651189 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648266 2535 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 10:27:40.651189 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648269 2535 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 10:27:40.651189 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648271 2535 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 10:27:40.651189 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648274 2535 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 10:27:40.651189 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648276 2535 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 10:27:40.651189 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648279 2535 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 10:27:40.651189 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648282 2535 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 10:27:40.651705 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648285 2535 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 10:27:40.651705 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648287 2535 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 10:27:40.651705 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648289 2535 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 10:27:40.651705 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648293 2535 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 10:27:40.651705 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648297 2535 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 10:27:40.651705 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648299 2535 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 10:27:40.651705 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648302 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 10:27:40.651705 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648305 2535 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 10:27:40.651705 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648308 2535 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 10:27:40.651705 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648311 2535 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 10:27:40.651705 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648313 2535 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 10:27:40.651705 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648316 2535 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 10:27:40.651705 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648318 2535 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 10:27:40.651705 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648322 2535 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 10:27:40.651705 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648667 2535 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 10:27:40.651705 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648673 2535 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 10:27:40.651705 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648677 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 10:27:40.651705 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648680 2535 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 10:27:40.652157 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648684 2535 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 10:27:40.652157 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648687 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 10:27:40.652157 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648690 2535 feature_gate.go:328] unrecognized feature gate: Example Aug 04 10:27:40.652157 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648693 2535 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 10:27:40.652157 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648696 2535 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 10:27:40.652157 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648698 2535 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 10:27:40.652157 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648701 2535 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 10:27:40.652157 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648704 2535 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 10:27:40.652157 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648706 2535 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 10:27:40.652157 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648709 2535 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 10:27:40.652157 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648711 2535 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 10:27:40.652157 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648714 2535 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 10:27:40.652157 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648718 2535 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 10:27:40.652157 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648720 2535 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 10:27:40.652157 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648722 2535 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 10:27:40.652157 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648725 2535 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 10:27:40.652157 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648727 2535 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 10:27:40.652157 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648730 2535 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 10:27:40.652157 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648734 2535 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 10:27:40.652157 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648737 2535 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 10:27:40.652654 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648739 2535 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 10:27:40.652654 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648743 2535 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 10:27:40.652654 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648745 2535 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 10:27:40.652654 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648748 2535 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 10:27:40.652654 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648750 2535 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 10:27:40.652654 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648753 2535 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 10:27:40.652654 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648755 2535 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 10:27:40.652654 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648758 2535 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 10:27:40.652654 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648761 2535 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 10:27:40.652654 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648764 2535 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 10:27:40.652654 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648766 2535 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 10:27:40.652654 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648769 2535 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 10:27:40.652654 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648772 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 10:27:40.652654 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648774 2535 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 10:27:40.652654 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648777 2535 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 10:27:40.652654 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648779 2535 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 10:27:40.652654 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648782 2535 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 10:27:40.652654 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648784 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 10:27:40.652654 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648787 2535 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 10:27:40.652654 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648789 2535 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 10:27:40.653159 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648792 2535 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 10:27:40.653159 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648794 2535 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 10:27:40.653159 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648797 2535 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 10:27:40.653159 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648799 2535 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 10:27:40.653159 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648802 2535 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 10:27:40.653159 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648805 2535 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 10:27:40.653159 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648807 2535 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 10:27:40.653159 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648810 2535 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 10:27:40.653159 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648812 2535 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 10:27:40.653159 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648815 2535 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 10:27:40.653159 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648817 2535 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 10:27:40.653159 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648820 2535 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 10:27:40.653159 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648822 2535 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 10:27:40.653159 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648825 2535 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 10:27:40.653159 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648828 2535 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 10:27:40.653159 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648831 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 10:27:40.653159 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648833 2535 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 10:27:40.653159 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648836 2535 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 10:27:40.653159 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648838 2535 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 10:27:40.653159 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648841 2535 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 10:27:40.653651 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648843 2535 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 10:27:40.653651 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648846 2535 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 10:27:40.653651 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648848 2535 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 10:27:40.653651 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648851 2535 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 10:27:40.653651 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648853 2535 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 10:27:40.653651 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648856 2535 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 10:27:40.653651 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648858 2535 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 10:27:40.653651 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648861 2535 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 10:27:40.653651 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648863 2535 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 10:27:40.653651 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648866 2535 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 10:27:40.653651 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648870 2535 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 10:27:40.653651 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648872 2535 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 10:27:40.653651 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648874 2535 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 10:27:40.653651 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648877 2535 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 10:27:40.653651 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648879 2535 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 10:27:40.653651 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648882 2535 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 10:27:40.653651 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648884 2535 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 10:27:40.653651 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648886 2535 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 10:27:40.653651 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648889 2535 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 10:27:40.653651 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648892 2535 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 10:27:40.654262 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648894 2535 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 10:27:40.654262 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648896 2535 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 10:27:40.654262 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648899 2535 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 10:27:40.654262 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648901 2535 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 10:27:40.654262 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648905 2535 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 10:27:40.654262 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648908 2535 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 10:27:40.654262 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648911 2535 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 10:27:40.654262 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648914 2535 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 10:27:40.654262 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648917 2535 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 10:27:40.654262 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648920 2535 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 10:27:40.654262 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648922 2535 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 10:27:40.654262 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648925 2535 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 10:27:40.654262 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648928 2535 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 10:27:40.654262 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648932 2535 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 10:27:40.654262 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.648935 2535 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 10:27:40.654262 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649001 2535 flags.go:64] FLAG: --address="0.0.0.0" Aug 04 10:27:40.654262 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649007 2535 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Aug 04 10:27:40.654262 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649013 2535 flags.go:64] FLAG: --anonymous-auth="true" Aug 04 10:27:40.654262 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649019 2535 flags.go:64] FLAG: --application-metrics-count-limit="100" Aug 04 10:27:40.654262 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649025 2535 flags.go:64] FLAG: --authentication-token-webhook="false" Aug 04 10:27:40.654752 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649030 2535 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Aug 04 10:27:40.654752 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649036 2535 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Aug 04 10:27:40.654752 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649041 2535 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Aug 04 10:27:40.654752 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649044 2535 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Aug 04 10:27:40.654752 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649047 2535 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Aug 04 10:27:40.654752 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649050 2535 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Aug 04 10:27:40.654752 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649056 2535 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Aug 04 10:27:40.654752 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649059 2535 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Aug 04 10:27:40.654752 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649062 2535 flags.go:64] FLAG: --cgroup-root="" Aug 04 10:27:40.654752 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649080 2535 flags.go:64] FLAG: --cgroups-per-qos="true" Aug 04 10:27:40.654752 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649084 2535 flags.go:64] FLAG: --client-ca-file="" Aug 04 10:27:40.654752 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649088 2535 flags.go:64] FLAG: --cloud-provider="external" Aug 04 10:27:40.654752 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649091 2535 flags.go:64] FLAG: --cluster-dns="[]" Aug 04 10:27:40.654752 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649095 2535 flags.go:64] FLAG: --cluster-domain="" Aug 04 10:27:40.654752 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649098 2535 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Aug 04 10:27:40.654752 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649101 2535 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Aug 04 10:27:40.654752 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649104 2535 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Aug 04 10:27:40.654752 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649107 2535 flags.go:64] FLAG: --container-log-max-files="5" Aug 04 10:27:40.654752 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649111 2535 flags.go:64] FLAG: --container-log-max-size="10Mi" Aug 04 10:27:40.654752 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649114 2535 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Aug 04 10:27:40.654752 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649117 2535 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Aug 04 10:27:40.654752 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649120 2535 flags.go:64] FLAG: --containerd-namespace="k8s.io" Aug 04 10:27:40.654752 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649123 2535 flags.go:64] FLAG: --contention-profiling="false" Aug 04 10:27:40.654752 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649126 2535 flags.go:64] FLAG: --cpu-cfs-quota="true" Aug 04 10:27:40.655330 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649129 2535 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Aug 04 10:27:40.655330 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649135 2535 flags.go:64] FLAG: --cpu-manager-policy="none" Aug 04 10:27:40.655330 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649138 2535 flags.go:64] FLAG: --cpu-manager-policy-options="" Aug 04 10:27:40.655330 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649142 2535 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Aug 04 10:27:40.655330 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649145 2535 flags.go:64] FLAG: --enable-controller-attach-detach="true" Aug 04 10:27:40.655330 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649148 2535 flags.go:64] FLAG: --enable-debugging-handlers="true" Aug 04 10:27:40.655330 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649151 2535 flags.go:64] FLAG: --enable-load-reader="false" Aug 04 10:27:40.655330 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649153 2535 flags.go:64] FLAG: --enable-server="true" Aug 04 10:27:40.655330 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649156 2535 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Aug 04 10:27:40.655330 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649160 2535 flags.go:64] FLAG: --event-burst="100" Aug 04 10:27:40.655330 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649163 2535 flags.go:64] FLAG: --event-qps="50" Aug 04 10:27:40.655330 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649166 2535 flags.go:64] FLAG: --event-storage-age-limit="default=0" Aug 04 10:27:40.655330 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649169 2535 flags.go:64] FLAG: --event-storage-event-limit="default=0" Aug 04 10:27:40.655330 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649171 2535 flags.go:64] FLAG: --eviction-hard="" Aug 04 10:27:40.655330 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649177 2535 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Aug 04 10:27:40.655330 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649179 2535 flags.go:64] FLAG: --eviction-minimum-reclaim="" Aug 04 10:27:40.655330 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649182 2535 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Aug 04 10:27:40.655330 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649185 2535 flags.go:64] FLAG: --eviction-soft="" Aug 04 10:27:40.655330 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649188 2535 flags.go:64] FLAG: --eviction-soft-grace-period="" Aug 04 10:27:40.655330 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649191 2535 flags.go:64] FLAG: --exit-on-lock-contention="false" Aug 04 10:27:40.655330 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649197 2535 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Aug 04 10:27:40.655330 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649200 2535 flags.go:64] FLAG: --experimental-mounter-path="" Aug 04 10:27:40.655330 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649203 2535 flags.go:64] FLAG: --fail-cgroupv1="false" Aug 04 10:27:40.655330 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649206 2535 flags.go:64] FLAG: --fail-swap-on="true" Aug 04 10:27:40.655330 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649208 2535 flags.go:64] FLAG: --feature-gates="" Aug 04 10:27:40.655982 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649212 2535 flags.go:64] FLAG: --file-check-frequency="20s" Aug 04 10:27:40.655982 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649215 2535 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Aug 04 10:27:40.655982 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649218 2535 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Aug 04 10:27:40.655982 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649221 2535 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Aug 04 10:27:40.655982 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649224 2535 flags.go:64] FLAG: --healthz-port="10248" Aug 04 10:27:40.655982 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649228 2535 flags.go:64] FLAG: --help="false" Aug 04 10:27:40.655982 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649231 2535 flags.go:64] FLAG: --hostname-override="ip-10-0-130-250.ec2.internal" Aug 04 10:27:40.655982 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649234 2535 flags.go:64] FLAG: --housekeeping-interval="10s" Aug 04 10:27:40.655982 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649238 2535 flags.go:64] FLAG: --http-check-frequency="20s" Aug 04 10:27:40.655982 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649241 2535 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Aug 04 10:27:40.655982 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649244 2535 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Aug 04 10:27:40.655982 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649247 2535 flags.go:64] FLAG: --image-gc-high-threshold="85" Aug 04 10:27:40.655982 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649250 2535 flags.go:64] FLAG: --image-gc-low-threshold="80" Aug 04 10:27:40.655982 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649253 2535 flags.go:64] FLAG: --image-service-endpoint="" Aug 04 10:27:40.655982 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649256 2535 flags.go:64] FLAG: --kernel-memcg-notification="false" Aug 04 10:27:40.655982 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649258 2535 flags.go:64] FLAG: --kube-api-burst="100" Aug 04 10:27:40.655982 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649261 2535 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Aug 04 10:27:40.655982 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649264 2535 flags.go:64] FLAG: --kube-api-qps="50" Aug 04 10:27:40.655982 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649267 2535 flags.go:64] FLAG: --kube-reserved="" Aug 04 10:27:40.655982 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649270 2535 flags.go:64] FLAG: --kube-reserved-cgroup="" Aug 04 10:27:40.655982 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649273 2535 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Aug 04 10:27:40.655982 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649277 2535 flags.go:64] FLAG: --kubelet-cgroups="" Aug 04 10:27:40.655982 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649279 2535 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Aug 04 10:27:40.655982 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649282 2535 flags.go:64] FLAG: --lock-file="" Aug 04 10:27:40.656620 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649285 2535 flags.go:64] FLAG: --log-cadvisor-usage="false" Aug 04 10:27:40.656620 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649288 2535 flags.go:64] FLAG: --log-flush-frequency="5s" Aug 04 10:27:40.656620 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649304 2535 flags.go:64] FLAG: --log-json-info-buffer-size="0" Aug 04 10:27:40.656620 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649310 2535 flags.go:64] FLAG: --log-json-split-stream="false" Aug 04 10:27:40.656620 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649314 2535 flags.go:64] FLAG: --log-text-info-buffer-size="0" Aug 04 10:27:40.656620 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649318 2535 flags.go:64] FLAG: --log-text-split-stream="false" Aug 04 10:27:40.656620 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649321 2535 flags.go:64] FLAG: --logging-format="text" Aug 04 10:27:40.656620 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649324 2535 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Aug 04 10:27:40.656620 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649327 2535 flags.go:64] FLAG: --make-iptables-util-chains="true" Aug 04 10:27:40.656620 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649330 2535 flags.go:64] FLAG: --manifest-url="" Aug 04 10:27:40.656620 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649332 2535 flags.go:64] FLAG: --manifest-url-header="" Aug 04 10:27:40.656620 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649336 2535 flags.go:64] FLAG: --max-housekeeping-interval="15s" Aug 04 10:27:40.656620 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649340 2535 flags.go:64] FLAG: --max-open-files="1000000" Aug 04 10:27:40.656620 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649344 2535 flags.go:64] FLAG: --max-pods="110" Aug 04 10:27:40.656620 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649346 2535 flags.go:64] FLAG: --maximum-dead-containers="-1" Aug 04 10:27:40.656620 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649349 2535 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Aug 04 10:27:40.656620 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649353 2535 flags.go:64] FLAG: --memory-manager-policy="None" Aug 04 10:27:40.656620 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649356 2535 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Aug 04 10:27:40.656620 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649359 2535 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Aug 04 10:27:40.656620 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649362 2535 flags.go:64] FLAG: --node-ip="0.0.0.0" Aug 04 10:27:40.656620 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649365 2535 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Aug 04 10:27:40.656620 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649372 2535 flags.go:64] FLAG: --node-status-max-images="50" Aug 04 10:27:40.656620 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649375 2535 flags.go:64] FLAG: --node-status-update-frequency="10s" Aug 04 10:27:40.656620 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649378 2535 flags.go:64] FLAG: --oom-score-adj="-999" Aug 04 10:27:40.656620 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649381 2535 flags.go:64] FLAG: --pod-cidr="" Aug 04 10:27:40.657272 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649384 2535 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8" Aug 04 10:27:40.657272 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649388 2535 flags.go:64] FLAG: --pod-manifest-path="" Aug 04 10:27:40.657272 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649391 2535 flags.go:64] FLAG: --pod-max-pids="-1" Aug 04 10:27:40.657272 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649394 2535 flags.go:64] FLAG: --pods-per-core="0" Aug 04 10:27:40.657272 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649398 2535 flags.go:64] FLAG: --port="10250" Aug 04 10:27:40.657272 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649401 2535 flags.go:64] FLAG: --protect-kernel-defaults="false" Aug 04 10:27:40.657272 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649404 2535 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0efd0f2586d8c54be" Aug 04 10:27:40.657272 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649407 2535 flags.go:64] FLAG: --qos-reserved="" Aug 04 10:27:40.657272 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649410 2535 flags.go:64] FLAG: --read-only-port="10255" Aug 04 10:27:40.657272 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649413 2535 flags.go:64] FLAG: --register-node="true" Aug 04 10:27:40.657272 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649416 2535 flags.go:64] FLAG: --register-with-taints="" Aug 04 10:27:40.657272 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649420 2535 flags.go:64] FLAG: --registry-burst="10" Aug 04 10:27:40.657272 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649423 2535 flags.go:64] FLAG: --registry-qps="5" Aug 04 10:27:40.657272 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649426 2535 flags.go:64] FLAG: --reserved-cpus="" Aug 04 10:27:40.657272 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649429 2535 flags.go:64] FLAG: --reserved-memory="" Aug 04 10:27:40.657272 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649436 2535 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Aug 04 10:27:40.657272 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649439 2535 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Aug 04 10:27:40.657272 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649442 2535 flags.go:64] FLAG: --rotate-certificates="false" Aug 04 10:27:40.657272 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649445 2535 flags.go:64] FLAG: --rotate-server-certificates="false" Aug 04 10:27:40.657272 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649448 2535 flags.go:64] FLAG: --runonce="false" Aug 04 10:27:40.657272 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649450 2535 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Aug 04 10:27:40.657272 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649453 2535 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Aug 04 10:27:40.657272 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649456 2535 flags.go:64] FLAG: --seccomp-default="false" Aug 04 10:27:40.657272 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649459 2535 flags.go:64] FLAG: --serialize-image-pulls="true" Aug 04 10:27:40.657272 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649463 2535 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Aug 04 10:27:40.657869 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649465 2535 flags.go:64] FLAG: --storage-driver-db="cadvisor" Aug 04 10:27:40.657869 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649469 2535 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Aug 04 10:27:40.657869 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649472 2535 flags.go:64] FLAG: --storage-driver-password="root" Aug 04 10:27:40.657869 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649475 2535 flags.go:64] FLAG: --storage-driver-secure="false" Aug 04 10:27:40.657869 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649478 2535 flags.go:64] FLAG: --storage-driver-table="stats" Aug 04 10:27:40.657869 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649480 2535 flags.go:64] FLAG: --storage-driver-user="root" Aug 04 10:27:40.657869 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649483 2535 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Aug 04 10:27:40.657869 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649486 2535 flags.go:64] FLAG: --sync-frequency="1m0s" Aug 04 10:27:40.657869 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649489 2535 flags.go:64] FLAG: --system-cgroups="" Aug 04 10:27:40.657869 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649492 2535 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Aug 04 10:27:40.657869 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649497 2535 flags.go:64] FLAG: --system-reserved-cgroup="" Aug 04 10:27:40.657869 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649501 2535 flags.go:64] FLAG: --tls-cert-file="" Aug 04 10:27:40.657869 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649504 2535 flags.go:64] FLAG: --tls-cipher-suites="[]" Aug 04 10:27:40.657869 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649508 2535 flags.go:64] FLAG: --tls-min-version="" Aug 04 10:27:40.657869 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649511 2535 flags.go:64] FLAG: --tls-private-key-file="" Aug 04 10:27:40.657869 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649513 2535 flags.go:64] FLAG: --topology-manager-policy="none" Aug 04 10:27:40.657869 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649516 2535 flags.go:64] FLAG: --topology-manager-policy-options="" Aug 04 10:27:40.657869 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649519 2535 flags.go:64] FLAG: --topology-manager-scope="container" Aug 04 10:27:40.657869 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649522 2535 flags.go:64] FLAG: --v="2" Aug 04 10:27:40.657869 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649526 2535 flags.go:64] FLAG: --version="false" Aug 04 10:27:40.657869 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649530 2535 flags.go:64] FLAG: --vmodule="" Aug 04 10:27:40.657869 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649533 2535 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Aug 04 10:27:40.657869 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.649537 2535 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Aug 04 10:27:40.657869 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649642 2535 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 10:27:40.657869 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649647 2535 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 10:27:40.658494 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649650 2535 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 10:27:40.658494 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649653 2535 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 10:27:40.658494 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649656 2535 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 10:27:40.658494 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649659 2535 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 10:27:40.658494 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649662 2535 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 10:27:40.658494 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649664 2535 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 10:27:40.658494 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649668 2535 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 10:27:40.658494 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649671 2535 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 10:27:40.658494 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649674 2535 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 10:27:40.658494 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649677 2535 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 10:27:40.658494 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649680 2535 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 10:27:40.658494 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649683 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 10:27:40.658494 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649686 2535 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 10:27:40.658494 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649688 2535 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 10:27:40.658494 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649692 2535 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 10:27:40.658494 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649696 2535 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 10:27:40.658494 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649699 2535 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 10:27:40.658494 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649701 2535 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 10:27:40.658494 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649705 2535 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 10:27:40.658955 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649707 2535 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 10:27:40.658955 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649710 2535 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 10:27:40.658955 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649713 2535 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 10:27:40.658955 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649715 2535 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 10:27:40.658955 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649718 2535 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 10:27:40.658955 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649721 2535 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 10:27:40.658955 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649724 2535 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 10:27:40.658955 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649728 2535 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 10:27:40.658955 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649731 2535 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 10:27:40.658955 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649735 2535 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 10:27:40.658955 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649738 2535 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 10:27:40.658955 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649740 2535 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 10:27:40.658955 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649743 2535 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 10:27:40.658955 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649745 2535 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 10:27:40.658955 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649748 2535 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 10:27:40.658955 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649750 2535 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 10:27:40.658955 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649753 2535 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 10:27:40.658955 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649755 2535 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 10:27:40.658955 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649758 2535 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 10:27:40.659425 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649762 2535 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 10:27:40.659425 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649765 2535 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 10:27:40.659425 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649767 2535 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 10:27:40.659425 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649770 2535 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 10:27:40.659425 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649774 2535 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 10:27:40.659425 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649778 2535 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 10:27:40.659425 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649780 2535 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 10:27:40.659425 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649783 2535 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 10:27:40.659425 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649785 2535 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 10:27:40.659425 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649788 2535 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 10:27:40.659425 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649791 2535 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 10:27:40.659425 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649793 2535 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 10:27:40.659425 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649797 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 10:27:40.659425 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649800 2535 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 10:27:40.659425 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649803 2535 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 10:27:40.659425 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649805 2535 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 10:27:40.659425 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649808 2535 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 10:27:40.659425 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649810 2535 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 10:27:40.659425 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649813 2535 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 10:27:40.659425 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649815 2535 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 10:27:40.659918 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649818 2535 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 10:27:40.659918 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649821 2535 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 10:27:40.659918 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649823 2535 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 10:27:40.659918 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649827 2535 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 10:27:40.659918 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649829 2535 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 10:27:40.659918 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649832 2535 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 10:27:40.659918 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649834 2535 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 10:27:40.659918 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649837 2535 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 10:27:40.659918 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649840 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 10:27:40.659918 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649842 2535 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 10:27:40.659918 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649845 2535 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 10:27:40.659918 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649847 2535 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 10:27:40.659918 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649851 2535 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 10:27:40.659918 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649854 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 10:27:40.659918 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649857 2535 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 10:27:40.659918 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649860 2535 feature_gate.go:328] unrecognized feature gate: Example Aug 04 10:27:40.659918 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649862 2535 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 10:27:40.659918 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649865 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 10:27:40.659918 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649867 2535 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 10:27:40.659918 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649870 2535 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 10:27:40.660414 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649873 2535 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 10:27:40.660414 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649876 2535 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 10:27:40.660414 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649878 2535 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 10:27:40.660414 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649881 2535 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 10:27:40.660414 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649885 2535 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 10:27:40.660414 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649887 2535 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 10:27:40.660414 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649890 2535 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 10:27:40.660414 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649892 2535 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 10:27:40.660414 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649894 2535 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 10:27:40.660414 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649897 2535 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 10:27:40.660414 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649899 2535 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 10:27:40.660414 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649902 2535 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 10:27:40.660414 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649904 2535 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 10:27:40.660414 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649907 2535 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 10:27:40.660414 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649909 2535 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 10:27:40.660414 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649912 2535 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 10:27:40.660414 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649915 2535 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 10:27:40.660414 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649917 2535 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 10:27:40.660414 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.649920 2535 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 10:27:40.660908 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.650543 2535 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 10:27:40.660908 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.656955 2535 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Aug 04 10:27:40.660908 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.656970 2535 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Aug 04 10:27:40.660908 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.657002 2535 watchdog_linux.go:95] "Systemd watchdog is not enabled" Aug 04 10:27:40.660908 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.657008 2535 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Aug 04 10:27:40.660908 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657088 2535 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 10:27:40.660908 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657093 2535 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 10:27:40.660908 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657097 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 10:27:40.660908 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657099 2535 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 10:27:40.660908 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657102 2535 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 10:27:40.660908 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657105 2535 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 10:27:40.660908 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657108 2535 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 10:27:40.660908 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657111 2535 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 10:27:40.660908 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657113 2535 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 10:27:40.660908 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657116 2535 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 10:27:40.661373 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657119 2535 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 10:27:40.661373 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657121 2535 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 10:27:40.661373 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657124 2535 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 10:27:40.661373 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657126 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 10:27:40.661373 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657129 2535 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 10:27:40.661373 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657132 2535 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 10:27:40.661373 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657136 2535 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 10:27:40.661373 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657139 2535 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 10:27:40.661373 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657143 2535 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 10:27:40.661373 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657145 2535 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 10:27:40.661373 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657148 2535 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 10:27:40.661373 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657151 2535 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 10:27:40.661373 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657154 2535 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 10:27:40.661373 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657157 2535 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 10:27:40.661373 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657165 2535 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 10:27:40.661373 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657168 2535 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 10:27:40.661373 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657171 2535 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 10:27:40.661373 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657174 2535 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 10:27:40.661373 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657177 2535 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 10:27:40.661838 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657180 2535 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 10:27:40.661838 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657183 2535 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 10:27:40.661838 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657186 2535 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 10:27:40.661838 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657188 2535 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 10:27:40.661838 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657191 2535 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 10:27:40.661838 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657194 2535 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 10:27:40.661838 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657196 2535 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 10:27:40.661838 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657199 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 10:27:40.661838 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657202 2535 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 10:27:40.661838 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657204 2535 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 10:27:40.661838 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657207 2535 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 10:27:40.661838 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657209 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 10:27:40.661838 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657213 2535 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 10:27:40.661838 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657217 2535 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 10:27:40.661838 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657220 2535 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 10:27:40.661838 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657223 2535 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 10:27:40.661838 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657225 2535 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 10:27:40.661838 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657228 2535 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 10:27:40.661838 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657231 2535 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 10:27:40.661838 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657234 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 10:27:40.662335 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657237 2535 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 10:27:40.662335 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657239 2535 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 10:27:40.662335 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657243 2535 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 10:27:40.662335 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657246 2535 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 10:27:40.662335 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657249 2535 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 10:27:40.662335 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657251 2535 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 10:27:40.662335 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657254 2535 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 10:27:40.662335 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657256 2535 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 10:27:40.662335 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657264 2535 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 10:27:40.662335 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657267 2535 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 10:27:40.662335 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657270 2535 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 10:27:40.662335 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657273 2535 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 10:27:40.662335 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657276 2535 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 10:27:40.662335 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657279 2535 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 10:27:40.662335 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657281 2535 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 10:27:40.662335 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657284 2535 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 10:27:40.662335 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657287 2535 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 10:27:40.662335 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657290 2535 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 10:27:40.662335 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657293 2535 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 10:27:40.662799 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657295 2535 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 10:27:40.662799 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657298 2535 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 10:27:40.662799 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657300 2535 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 10:27:40.662799 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657303 2535 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 10:27:40.662799 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657305 2535 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 10:27:40.662799 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657307 2535 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 10:27:40.662799 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657310 2535 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 10:27:40.662799 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657312 2535 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 10:27:40.662799 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657315 2535 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 10:27:40.662799 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657317 2535 feature_gate.go:328] unrecognized feature gate: Example Aug 04 10:27:40.662799 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657320 2535 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 10:27:40.662799 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657322 2535 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 10:27:40.662799 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657325 2535 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 10:27:40.662799 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657327 2535 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 10:27:40.662799 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657330 2535 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 10:27:40.662799 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657332 2535 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 10:27:40.662799 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657335 2535 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 10:27:40.662799 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657337 2535 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 10:27:40.662799 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657340 2535 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 10:27:40.662799 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657342 2535 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 10:27:40.663284 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657345 2535 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 10:27:40.663284 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657347 2535 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 10:27:40.663284 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657356 2535 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 10:27:40.663284 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657360 2535 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 10:27:40.663284 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657362 2535 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 10:27:40.663284 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657365 2535 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 10:27:40.663284 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657367 2535 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 10:27:40.663284 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657370 2535 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 10:27:40.663284 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657373 2535 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 10:27:40.663284 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657375 2535 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 10:27:40.663284 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657378 2535 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 10:27:40.663284 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.657383 2535 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 10:27:40.663284 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657516 2535 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 10:27:40.663284 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657520 2535 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 10:27:40.663284 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657523 2535 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 10:27:40.663647 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657526 2535 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 10:27:40.663647 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657529 2535 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 10:27:40.663647 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657532 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 10:27:40.663647 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657534 2535 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 10:27:40.663647 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657537 2535 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 10:27:40.663647 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657539 2535 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 10:27:40.663647 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657543 2535 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 10:27:40.663647 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657546 2535 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 10:27:40.663647 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657550 2535 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 10:27:40.663647 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657552 2535 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 10:27:40.663647 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657555 2535 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 10:27:40.663647 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657558 2535 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 10:27:40.663647 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657561 2535 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 10:27:40.663647 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657564 2535 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 10:27:40.663647 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657567 2535 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 10:27:40.663647 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657570 2535 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 10:27:40.663647 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657573 2535 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 10:27:40.663647 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657576 2535 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 10:27:40.663647 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657578 2535 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 10:27:40.664152 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657581 2535 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 10:27:40.664152 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657590 2535 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 10:27:40.664152 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657594 2535 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 10:27:40.664152 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657601 2535 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 10:27:40.664152 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657604 2535 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 10:27:40.664152 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657607 2535 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 10:27:40.664152 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657610 2535 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 10:27:40.664152 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657613 2535 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 10:27:40.664152 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657616 2535 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 10:27:40.664152 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657619 2535 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 10:27:40.664152 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657622 2535 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 10:27:40.664152 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657625 2535 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 10:27:40.664152 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657627 2535 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 10:27:40.664152 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657629 2535 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 10:27:40.664152 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657632 2535 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 10:27:40.664152 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657635 2535 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 10:27:40.664152 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657637 2535 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 10:27:40.664152 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657640 2535 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 10:27:40.664152 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657642 2535 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 10:27:40.664152 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657645 2535 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 10:27:40.664626 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657648 2535 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 10:27:40.664626 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657650 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 10:27:40.664626 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657653 2535 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 10:27:40.664626 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657655 2535 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 10:27:40.664626 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657658 2535 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 10:27:40.664626 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657660 2535 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 10:27:40.664626 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657663 2535 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 10:27:40.664626 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657665 2535 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 10:27:40.664626 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657667 2535 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 10:27:40.664626 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657670 2535 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 10:27:40.664626 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657672 2535 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 10:27:40.664626 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657675 2535 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 10:27:40.664626 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657677 2535 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 10:27:40.664626 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657680 2535 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 10:27:40.664626 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657688 2535 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 10:27:40.664626 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657691 2535 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 10:27:40.664626 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657694 2535 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 10:27:40.664626 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657696 2535 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 10:27:40.664626 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657698 2535 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 10:27:40.664626 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657701 2535 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 10:27:40.665109 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657703 2535 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 10:27:40.665109 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657706 2535 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 10:27:40.665109 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657708 2535 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 10:27:40.665109 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657711 2535 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 10:27:40.665109 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657713 2535 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 10:27:40.665109 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657716 2535 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 10:27:40.665109 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657718 2535 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 10:27:40.665109 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657721 2535 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 10:27:40.665109 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657723 2535 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 10:27:40.665109 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657726 2535 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 10:27:40.665109 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657728 2535 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 10:27:40.665109 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657731 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 10:27:40.665109 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657733 2535 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 10:27:40.665109 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657736 2535 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 10:27:40.665109 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657738 2535 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 10:27:40.665109 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657741 2535 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 10:27:40.665109 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657743 2535 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 10:27:40.665109 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657746 2535 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 10:27:40.665109 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657748 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 10:27:40.665109 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657751 2535 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 10:27:40.665576 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657753 2535 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 10:27:40.665576 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657756 2535 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 10:27:40.665576 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657758 2535 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 10:27:40.665576 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657761 2535 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 10:27:40.665576 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657763 2535 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 10:27:40.665576 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657766 2535 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 10:27:40.665576 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657768 2535 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 10:27:40.665576 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657777 2535 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 10:27:40.665576 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657779 2535 feature_gate.go:328] unrecognized feature gate: Example Aug 04 10:27:40.665576 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657782 2535 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 10:27:40.665576 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657785 2535 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 10:27:40.665576 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657787 2535 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 10:27:40.665576 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657790 2535 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 10:27:40.665576 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657792 2535 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 10:27:40.665576 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657795 2535 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 10:27:40.665576 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657797 2535 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 10:27:40.665576 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:40.657799 2535 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 10:27:40.665968 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.657810 2535 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 10:27:40.665968 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.657914 2535 server.go:962] "Client rotation is on, will bootstrap in background" Aug 04 10:27:40.665968 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.659970 2535 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Aug 04 10:27:40.665968 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.661320 2535 server.go:1019] "Starting client certificate rotation" Aug 04 10:27:40.665968 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.661428 2535 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Aug 04 10:27:40.665968 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.662662 2535 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Aug 04 10:27:40.665968 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.664445 2535 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Aug 04 10:27:40.672059 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.672038 2535 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Aug 04 10:27:40.680926 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.680910 2535 log.go:25] "Validated CRI v1 runtime API" Aug 04 10:27:40.682866 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.682851 2535 log.go:25] "Validated CRI v1 image API" Aug 04 10:27:40.685570 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.685553 2535 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Aug 04 10:27:40.687835 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.687805 2535 fs.go:135] Filesystem UUIDs: map[0cc5e985-4e65-4b2b-bb83-f3bd1f752da2:/dev/nvme0n1p4 5fc98a5e-2c33-4113-87ed-0be1cd0509b7:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Aug 04 10:27:40.687920 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.687832 2535 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Aug 04 10:27:40.689731 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.689710 2535 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Aug 04 10:27:40.693317 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.693207 2535 manager.go:216] Machine: {Timestamp:2026-08-04 10:27:40.691982913 +0000 UTC m=+0.317483089 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098049 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c2bcae6f4fb3fd67860ac5057e88f SystemUUID:ec2c2bca-e6f4-fb3f-d678-60ac5057e88f BootID:72170aa3-1af5-4441-9c7f-f49d256c234c Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3f:24:1a:b5:11 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3f:24:1a:b5:11 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:56:e6:0e:5d:07:74 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Aug 04 10:27:40.693317 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.693310 2535 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Aug 04 10:27:40.693474 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.693403 2535 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Aug 04 10:27:40.696000 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.695974 2535 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Aug 04 10:27:40.696191 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.696003 2535 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-250.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Aug 04 10:27:40.696869 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.696857 2535 topology_manager.go:138] "Creating topology manager with none policy" Aug 04 10:27:40.696926 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.696875 2535 container_manager_linux.go:309] "Creating device plugin manager" Aug 04 10:27:40.696926 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.696885 2535 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Aug 04 10:27:40.698108 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.698086 2535 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Aug 04 10:27:40.699183 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.699172 2535 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Aug 04 10:27:40.702264 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.702251 2535 state_mem.go:36] "Initialized new in-memory state store" Aug 04 10:27:40.702403 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.702392 2535 server.go:1240] "Using root directory" path="/var/lib/kubelet" Aug 04 10:27:40.702458 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.702427 2535 kubelet.go:486] "Attempting to sync node with API server" Aug 04 10:27:40.702458 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.702457 2535 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Aug 04 10:27:40.702546 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.702472 2535 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Aug 04 10:27:40.702546 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.702486 2535 kubelet.go:398] "Adding apiserver pod source" Aug 04 10:27:40.702546 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.702498 2535 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Aug 04 10:27:40.702963 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.702946 2535 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Aug 04 10:27:40.703025 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.702975 2535 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Aug 04 10:27:40.704991 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.704976 2535 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Aug 04 10:27:40.706370 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.706353 2535 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Aug 04 10:27:40.706410 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.706387 2535 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Aug 04 10:27:40.706571 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.706561 2535 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Aug 04 10:27:40.706604 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.706576 2535 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Aug 04 10:27:40.706604 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.706582 2535 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Aug 04 10:27:40.706604 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.706603 2535 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Aug 04 10:27:40.706689 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.706616 2535 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Aug 04 10:27:40.706689 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.706623 2535 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Aug 04 10:27:40.707095 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.707086 2535 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Aug 04 10:27:40.707095 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.707095 2535 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Aug 04 10:27:40.707154 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.707101 2535 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Aug 04 10:27:40.707154 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.707106 2535 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Aug 04 10:27:40.707154 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.707118 2535 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Aug 04 10:27:40.707154 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.707126 2535 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Aug 04 10:27:40.708289 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.708279 2535 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Aug 04 10:27:40.708289 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.708289 2535 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Aug 04 10:27:40.711461 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.711448 2535 server.go:1268] "Started kubelet" Aug 04 10:27:40.712201 ip-10-0-130-250 systemd[1]: Started Kubernetes Kubelet. Aug 04 10:27:40.712407 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.712232 2535 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Aug 04 10:27:40.714730 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.712469 2535 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Aug 04 10:27:40.714821 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.714763 2535 server_v1.go:49] "podresources" method="list" useActivePods=true Aug 04 10:27:40.716271 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.716247 2535 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Aug 04 10:27:40.716488 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.716471 2535 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9zk96" Aug 04 10:27:40.717555 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:40.717530 2535 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Aug 04 10:27:40.717644 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.717571 2535 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-250.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Aug 04 10:27:40.717644 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.717621 2535 server.go:310] "Adding debug handlers to kubelet server" Aug 04 10:27:40.717754 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:40.717666 2535 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-250.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Aug 04 10:27:40.719011 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:40.718991 2535 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Aug 04 10:27:40.722493 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:40.721520 2535 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-250.ec2.internal.18c8935888a6a69a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-250.ec2.internal,UID:ip-10-0-130-250.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-250.ec2.internal,},FirstTimestamp:2026-08-04 10:27:40.711429786 +0000 UTC m=+0.336929940,LastTimestamp:2026-08-04 10:27:40.711429786 +0000 UTC m=+0.336929940,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-250.ec2.internal,}" Aug 04 10:27:40.723911 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.723889 2535 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9zk96" Aug 04 10:27:40.724233 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.724218 2535 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Aug 04 10:27:40.724283 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.724226 2535 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Aug 04 10:27:40.724555 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:40.724373 2535 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-250.ec2.internal\" not found" Aug 04 10:27:40.724872 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.724851 2535 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Aug 04 10:27:40.725014 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.724999 2535 volume_manager.go:311] "The desired_state_of_world populator starts" Aug 04 10:27:40.725098 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.725017 2535 volume_manager.go:313] "Starting Kubelet Volume Manager" Aug 04 10:27:40.725172 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.725154 2535 reconstruct.go:97] "Volume reconstruction finished" Aug 04 10:27:40.725172 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.725168 2535 reconciler.go:29] "Reconciler: start to sync state" Aug 04 10:27:40.725495 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.725479 2535 factory.go:55] Registering systemd factory Aug 04 10:27:40.725495 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.725497 2535 factory.go:223] Registration of the systemd container factory successfully Aug 04 10:27:40.725790 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.725776 2535 factory.go:153] Registering CRI-O factory Aug 04 10:27:40.725855 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.725793 2535 factory.go:223] Registration of the crio container factory successfully Aug 04 10:27:40.725855 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.725837 2535 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Aug 04 10:27:40.725921 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.725857 2535 factory.go:103] Registering Raw factory Aug 04 10:27:40.725921 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.725881 2535 manager.go:1196] Started watching for new ooms in manager Aug 04 10:27:40.726384 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.726364 2535 manager.go:349] Starting recovery of all containers Aug 04 10:27:40.732854 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.732832 2535 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 10:27:40.733002 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.732982 2535 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Aug 04 10:27:40.736407 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.736388 2535 manager.go:354] Recovery completed Aug 04 10:27:40.736655 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:40.736621 2535 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-130-250.ec2.internal\" not found" node="ip-10-0-130-250.ec2.internal" Aug 04 10:27:40.740783 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.740771 2535 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 10:27:40.743513 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.743499 2535 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasSufficientMemory" Aug 04 10:27:40.743571 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.743525 2535 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 10:27:40.743571 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.743534 2535 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasSufficientPID" Aug 04 10:27:40.744006 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.743995 2535 cpu_manager.go:222] "Starting CPU manager" policy="none" Aug 04 10:27:40.744044 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.744006 2535 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Aug 04 10:27:40.744044 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.744021 2535 state_mem.go:36] "Initialized new in-memory state store" Aug 04 10:27:40.744124 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.744060 2535 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 10:27:40.745055 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.745045 2535 policy_none.go:49] "None policy: Start" Aug 04 10:27:40.745126 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.745059 2535 memory_manager.go:187] "Starting memorymanager" policy="None" Aug 04 10:27:40.745126 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.745099 2535 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Aug 04 10:27:40.745690 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.745673 2535 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasSufficientMemory" Aug 04 10:27:40.745775 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.745698 2535 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 10:27:40.745775 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.745711 2535 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasSufficientPID" Aug 04 10:27:40.746009 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.745995 2535 policy_none.go:47] "Start" Aug 04 10:27:40.790712 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.790590 2535 manager.go:337] "Starting Device Plugin manager" Aug 04 10:27:40.790810 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:40.790751 2535 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Aug 04 10:27:40.790810 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.790765 2535 server.go:87] "Starting device plugin registration server" Aug 04 10:27:40.790996 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.790984 2535 eviction_manager.go:189] "Eviction manager: starting control loop" Aug 04 10:27:40.791052 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.790997 2535 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Aug 04 10:27:40.791127 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.791110 2535 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Aug 04 10:27:40.791212 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.791191 2535 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Aug 04 10:27:40.791212 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.791201 2535 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Aug 04 10:27:40.791675 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:40.791655 2535 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Aug 04 10:27:40.791748 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:40.791729 2535 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-250.ec2.internal\" not found" Aug 04 10:27:40.821837 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.821820 2535 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Aug 04 10:27:40.823806 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.821855 2535 status_manager.go:244] "Starting to sync pod status with apiserver" Aug 04 10:27:40.823806 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.821874 2535 kubelet.go:2448] "Starting kubelet main sync loop" Aug 04 10:27:40.823806 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:40.821903 2535 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Aug 04 10:27:40.825920 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.825905 2535 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 10:27:40.891414 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.891368 2535 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 10:27:40.892229 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.892216 2535 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasSufficientMemory" Aug 04 10:27:40.892298 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.892242 2535 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 10:27:40.892298 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.892252 2535 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasSufficientPID" Aug 04 10:27:40.892298 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.892274 2535 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-250.ec2.internal" Aug 04 10:27:40.903574 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.903558 2535 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-250.ec2.internal" Aug 04 10:27:40.903619 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:40.903577 2535 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-250.ec2.internal\": node \"ip-10-0-130-250.ec2.internal\" not found" Aug 04 10:27:40.921875 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:40.921856 2535 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-250.ec2.internal\" not found" Aug 04 10:27:40.922823 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.922808 2535 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-250.ec2.internal"] Aug 04 10:27:40.922867 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.922862 2535 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 10:27:40.923592 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.923578 2535 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasSufficientMemory" Aug 04 10:27:40.923656 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.923604 2535 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 10:27:40.923656 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.923614 2535 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasSufficientPID" Aug 04 10:27:40.924785 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.924773 2535 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 10:27:40.924945 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.924931 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal" Aug 04 10:27:40.924990 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.924964 2535 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 10:27:40.925550 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.925538 2535 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasSufficientMemory" Aug 04 10:27:40.925609 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.925560 2535 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasSufficientMemory" Aug 04 10:27:40.925609 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.925566 2535 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 10:27:40.925609 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.925577 2535 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasSufficientPID" Aug 04 10:27:40.925609 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.925580 2535 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 10:27:40.925609 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.925591 2535 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasSufficientPID" Aug 04 10:27:40.926381 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.926365 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ce61b1a97eb01ff18ec04bf7a4d71f9c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal\" (UID: \"ce61b1a97eb01ff18ec04bf7a4d71f9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal" Aug 04 10:27:40.926433 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.926388 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce61b1a97eb01ff18ec04bf7a4d71f9c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal\" (UID: \"ce61b1a97eb01ff18ec04bf7a4d71f9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal" Aug 04 10:27:40.926433 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.926406 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7c39b691124fb3a351388d810da798f-config\") pod \"kube-apiserver-proxy-ip-10-0-130-250.ec2.internal\" (UID: \"b7c39b691124fb3a351388d810da798f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-250.ec2.internal" Aug 04 10:27:40.926723 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.926708 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-250.ec2.internal" Aug 04 10:27:40.926758 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.926741 2535 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 10:27:40.927392 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.927377 2535 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasSufficientMemory" Aug 04 10:27:40.927454 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.927408 2535 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 10:27:40.927454 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:40.927425 2535 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasSufficientPID" Aug 04 10:27:40.958655 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:40.958635 2535 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-250.ec2.internal\" not found" node="ip-10-0-130-250.ec2.internal" Aug 04 10:27:40.963053 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:40.963037 2535 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-250.ec2.internal\" not found" node="ip-10-0-130-250.ec2.internal" Aug 04 10:27:42.673962 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:42.673937 2535 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-250.ec2.internal\" not found" Aug 04 10:27:42.678470 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:42.678451 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7c39b691124fb3a351388d810da798f-config\") pod \"kube-apiserver-proxy-ip-10-0-130-250.ec2.internal\" (UID: \"b7c39b691124fb3a351388d810da798f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-250.ec2.internal" Aug 04 10:27:42.678577 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:42.678477 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ce61b1a97eb01ff18ec04bf7a4d71f9c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal\" (UID: \"ce61b1a97eb01ff18ec04bf7a4d71f9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal" Aug 04 10:27:42.678577 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:42.678522 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce61b1a97eb01ff18ec04bf7a4d71f9c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal\" (UID: \"ce61b1a97eb01ff18ec04bf7a4d71f9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal" Aug 04 10:27:42.678577 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:42.678555 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce61b1a97eb01ff18ec04bf7a4d71f9c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal\" (UID: \"ce61b1a97eb01ff18ec04bf7a4d71f9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal" Aug 04 10:27:42.678577 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:42.678553 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ce61b1a97eb01ff18ec04bf7a4d71f9c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal\" (UID: \"ce61b1a97eb01ff18ec04bf7a4d71f9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal" Aug 04 10:27:42.678577 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:42.678553 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7c39b691124fb3a351388d810da798f-config\") pod \"kube-apiserver-proxy-ip-10-0-130-250.ec2.internal\" (UID: \"b7c39b691124fb3a351388d810da798f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-250.ec2.internal" Aug 04 10:27:42.774116 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:42.774056 2535 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-250.ec2.internal\" not found" Aug 04 10:27:42.874845 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:42.874772 2535 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-250.ec2.internal\" not found" Aug 04 10:27:42.912209 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:42.912187 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal" Aug 04 10:27:42.917077 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:42.917017 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-250.ec2.internal" Aug 04 10:27:42.975057 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:42.975025 2535 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-250.ec2.internal\" not found" Aug 04 10:27:43.075553 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:43.075524 2535 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-250.ec2.internal\" not found" Aug 04 10:27:43.175987 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:43.175929 2535 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-250.ec2.internal\" not found" Aug 04 10:27:43.276430 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:43.276403 2535 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-250.ec2.internal\" not found" Aug 04 10:27:43.312699 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:43.312664 2535 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Aug 04 10:27:43.313098 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:43.312829 2535 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Aug 04 10:27:43.313098 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:43.312828 2535 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Aug 04 10:27:43.377295 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:43.377274 2535 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-250.ec2.internal\" not found" Aug 04 10:27:43.378904 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:43.378874 2535 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-03 10:22:42 +0000 UTC" deadline="2028-03-05 00:26:09.248200262 +0000 UTC" Aug 04 10:27:43.378904 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:43.378899 2535 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13885h58m25.869304086s" Aug 04 10:27:43.477695 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:43.477669 2535 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-250.ec2.internal\" not found" Aug 04 10:27:43.511475 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:43.511451 2535 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 10:27:43.544029 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:43.543991 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce61b1a97eb01ff18ec04bf7a4d71f9c.slice/crio-13b4b701502e797d1da9e7de4f19d774bba21cc6a645d457b62dbd15fe3dd209 WatchSource:0}: Error finding container 13b4b701502e797d1da9e7de4f19d774bba21cc6a645d457b62dbd15fe3dd209: Status 404 returned error can't find the container with id 13b4b701502e797d1da9e7de4f19d774bba21cc6a645d457b62dbd15fe3dd209 Aug 04 10:27:43.544762 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:43.544735 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7c39b691124fb3a351388d810da798f.slice/crio-3376ce2864934d6b3693fac5e488a7dac87364269053c5639b3d98037b6cdb62 WatchSource:0}: Error finding container 3376ce2864934d6b3693fac5e488a7dac87364269053c5639b3d98037b6cdb62: Status 404 returned error can't find the container with id 3376ce2864934d6b3693fac5e488a7dac87364269053c5639b3d98037b6cdb62 Aug 04 10:27:43.549521 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:43.549502 2535 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 10:27:43.576967 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:43.576945 2535 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-250.ec2.internal" Aug 04 10:27:43.585847 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:43.585830 2535 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Aug 04 10:27:43.587627 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:43.587615 2535 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal" Aug 04 10:27:43.600843 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:43.600824 2535 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Aug 04 10:27:43.900759 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:43.900730 2535 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 10:27:44.355101 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.355056 2535 apiserver.go:52] "Watching apiserver" Aug 04 10:27:44.361965 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.361943 2535 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Aug 04 10:27:44.364654 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.364624 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-zzlbv","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zcvc","openshift-multus/multus-additional-cni-plugins-s59fm","openshift-multus/network-metrics-daemon-qgpx8","openshift-network-operator/iptables-alerter-g4z5z","openshift-ovn-kubernetes/ovnkube-node-lcxhd","kube-system/konnectivity-agent-5bmt4","kube-system/kube-apiserver-proxy-ip-10-0-130-250.ec2.internal","openshift-cluster-node-tuning-operator/tuned-6xm6k","openshift-dns/node-resolver-s629f","openshift-image-registry/node-ca-zpdbz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal","openshift-multus/multus-6xqqk","openshift-network-diagnostics/network-check-target-2rc2x"] Aug 04 10:27:44.366096 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.366077 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5bmt4" Aug 04 10:27:44.367255 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.367236 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.368215 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.368194 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Aug 04 10:27:44.368556 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.368538 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-r7bpb\"" Aug 04 10:27:44.368645 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.368538 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Aug 04 10:27:44.368828 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.368813 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-s59fm" Aug 04 10:27:44.369393 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.369372 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Aug 04 10:27:44.369547 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.369529 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Aug 04 10:27:44.369656 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.369615 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Aug 04 10:27:44.369750 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.369384 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-tdbds\"" Aug 04 10:27:44.369814 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.369790 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Aug 04 10:27:44.370411 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.370393 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Aug 04 10:27:44.371032 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.370711 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ffgmx\"" Aug 04 10:27:44.371032 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.370788 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:27:44.371032 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.370822 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Aug 04 10:27:44.371032 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:44.370871 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgpx8" podUID="5436bc5b-0740-4ccb-b4fd-857d4023d8e7" Aug 04 10:27:44.372653 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.372170 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-g4z5z" Aug 04 10:27:44.373918 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.373399 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.374558 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.374240 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Aug 04 10:27:44.374558 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.374291 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Aug 04 10:27:44.374558 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.374465 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.374757 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.374689 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Aug 04 10:27:44.374836 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.374812 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-64vqm\"" Aug 04 10:27:44.375390 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.375373 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Aug 04 10:27:44.375661 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.375640 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Aug 04 10:27:44.375768 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.375756 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cjz64\"" Aug 04 10:27:44.375939 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.375924 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:27:44.376016 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:44.375976 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zzlbv" podUID="73272ce5-1238-42c7-b697-29d21e373fc0" Aug 04 10:27:44.376541 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.376462 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Aug 04 10:27:44.376635 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.376543 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Aug 04 10:27:44.376695 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.376640 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Aug 04 10:27:44.377038 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.376933 2535 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Aug 04 10:27:44.378021 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.377634 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Aug 04 10:27:44.378021 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.377807 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Aug 04 10:27:44.378021 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.377868 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pbb8t\"" Aug 04 10:27:44.378021 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.377994 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Aug 04 10:27:44.378021 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.378009 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zcvc" Aug 04 10:27:44.379198 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.379181 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s629f" Aug 04 10:27:44.379697 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.379679 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Aug 04 10:27:44.380553 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.380200 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Aug 04 10:27:44.380553 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.380343 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Aug 04 10:27:44.380553 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.380401 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sw94l\"" Aug 04 10:27:44.380765 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.380691 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zpdbz" Aug 04 10:27:44.381143 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.381117 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Aug 04 10:27:44.381341 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.381326 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zkpq5\"" Aug 04 10:27:44.381787 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.381687 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Aug 04 10:27:44.382399 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.382383 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Aug 04 10:27:44.382856 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.382834 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:27:44.382943 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:44.382898 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rc2x" podUID="b6f16f94-e1b9-4900-a75d-84badd1d9bb7" Aug 04 10:27:44.383349 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.383172 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2d8gs\"" Aug 04 10:27:44.383349 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.383225 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Aug 04 10:27:44.383828 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.383681 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Aug 04 10:27:44.384892 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.384872 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-etc-kubernetes\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.384977 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.384907 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-ovn-node-metrics-cert\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.384977 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.384932 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8c78f70d-8823-4d6c-96f8-c1318ceddd69-registration-dir\") pod \"aws-ebs-csi-driver-node-5zcvc\" (UID: \"8c78f70d-8823-4d6c-96f8-c1318ceddd69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zcvc" Aug 04 10:27:44.384977 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.384957 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7298c7a0-dc2b-4480-aa51-85ba014dcec6-cnibin\") pod \"multus-additional-cni-plugins-s59fm\" (UID: \"7298c7a0-dc2b-4480-aa51-85ba014dcec6\") " pod="openshift-multus/multus-additional-cni-plugins-s59fm" Aug 04 10:27:44.385098 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.384980 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dfe34fb7-de6d-42c3-b605-5015d4e822c7-iptables-alerter-script\") pod \"iptables-alerter-g4z5z\" (UID: \"dfe34fb7-de6d-42c3-b605-5015d4e822c7\") " pod="openshift-network-operator/iptables-alerter-g4z5z" Aug 04 10:27:44.385098 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.385005 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-host-run-netns\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.385098 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.385028 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-env-overrides\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.385098 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.385055 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8c78f70d-8823-4d6c-96f8-c1318ceddd69-etc-selinux\") pod \"aws-ebs-csi-driver-node-5zcvc\" (UID: \"8c78f70d-8823-4d6c-96f8-c1318ceddd69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zcvc" Aug 04 10:27:44.385098 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.385078 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8c78f70d-8823-4d6c-96f8-c1318ceddd69-sys-fs\") pod \"aws-ebs-csi-driver-node-5zcvc\" (UID: \"8c78f70d-8823-4d6c-96f8-c1318ceddd69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zcvc" Aug 04 10:27:44.385288 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.385111 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5dzf\" (UniqueName: \"kubernetes.io/projected/7298c7a0-dc2b-4480-aa51-85ba014dcec6-kube-api-access-h5dzf\") pod \"multus-additional-cni-plugins-s59fm\" (UID: \"7298c7a0-dc2b-4480-aa51-85ba014dcec6\") " pod="openshift-multus/multus-additional-cni-plugins-s59fm" Aug 04 10:27:44.385288 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.385134 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-host-run-k8s-cni-cncf-io\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.385288 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.385160 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-hostroot\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.385288 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.385182 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dfe34fb7-de6d-42c3-b605-5015d4e822c7-host-slash\") pod \"iptables-alerter-g4z5z\" (UID: \"dfe34fb7-de6d-42c3-b605-5015d4e822c7\") " pod="openshift-network-operator/iptables-alerter-g4z5z" Aug 04 10:27:44.385288 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.385203 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7298c7a0-dc2b-4480-aa51-85ba014dcec6-system-cni-dir\") pod \"multus-additional-cni-plugins-s59fm\" (UID: \"7298c7a0-dc2b-4480-aa51-85ba014dcec6\") " pod="openshift-multus/multus-additional-cni-plugins-s59fm" Aug 04 10:27:44.385288 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.385228 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7298c7a0-dc2b-4480-aa51-85ba014dcec6-os-release\") pod \"multus-additional-cni-plugins-s59fm\" (UID: \"7298c7a0-dc2b-4480-aa51-85ba014dcec6\") " pod="openshift-multus/multus-additional-cni-plugins-s59fm" Aug 04 10:27:44.385288 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.385251 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5436bc5b-0740-4ccb-b4fd-857d4023d8e7-metrics-certs\") pod \"network-metrics-daemon-qgpx8\" (UID: \"5436bc5b-0740-4ccb-b4fd-857d4023d8e7\") " pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:27:44.385288 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.385272 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-host-kubelet\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.385595 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.385295 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-run-ovn\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.385595 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.385317 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-log-socket\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.385595 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.385339 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-host-cni-netd\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.385595 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.385362 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-host-var-lib-cni-multus\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.385595 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.385384 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-host-slash\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.385822 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.385623 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-var-lib-openvswitch\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.385822 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.385680 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-etc-openvswitch\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.385822 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.385718 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-etc-modprobe-d\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.385822 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.385765 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-etc-sysconfig\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.385822 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.385797 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-etc-sysctl-conf\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.386037 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.385961 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-host-run-ovn-kubernetes\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.386037 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.385993 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-host-cni-bin\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.386130 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.386033 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.386130 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.386066 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-host-run-netns\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.386835 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.386254 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-multus-daemon-config\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.386835 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.386295 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-node-log\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.386835 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.386334 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-ovnkube-script-lib\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.386835 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.386515 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7298c7a0-dc2b-4480-aa51-85ba014dcec6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s59fm\" (UID: \"7298c7a0-dc2b-4480-aa51-85ba014dcec6\") " pod="openshift-multus/multus-additional-cni-plugins-s59fm" Aug 04 10:27:44.386835 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.386549 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-system-cni-dir\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.386835 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.386578 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-etc-systemd\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.387735 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.387710 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-run\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.387827 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.387756 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/73272ce5-1238-42c7-b697-29d21e373fc0-kubelet-config\") pod \"global-pull-secret-syncer-zzlbv\" (UID: \"73272ce5-1238-42c7-b697-29d21e373fc0\") " pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:27:44.387827 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.387801 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7298c7a0-dc2b-4480-aa51-85ba014dcec6-cni-binary-copy\") pod \"multus-additional-cni-plugins-s59fm\" (UID: \"7298c7a0-dc2b-4480-aa51-85ba014dcec6\") " pod="openshift-multus/multus-additional-cni-plugins-s59fm" Aug 04 10:27:44.388095 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.387830 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-multus-cni-dir\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.388095 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.387857 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-sys\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.388095 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.387910 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8c78f70d-8823-4d6c-96f8-c1318ceddd69-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5zcvc\" (UID: \"8c78f70d-8823-4d6c-96f8-c1318ceddd69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zcvc" Aug 04 10:27:44.388095 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.387943 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-host-var-lib-cni-bin\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.388095 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.387970 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr5nm\" (UniqueName: \"kubernetes.io/projected/5436bc5b-0740-4ccb-b4fd-857d4023d8e7-kube-api-access-zr5nm\") pod \"network-metrics-daemon-qgpx8\" (UID: \"5436bc5b-0740-4ccb-b4fd-857d4023d8e7\") " pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:27:44.388095 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.387996 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qdfx\" (UniqueName: \"kubernetes.io/projected/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-kube-api-access-8qdfx\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.388095 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.388026 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvf7r\" (UniqueName: \"kubernetes.io/projected/d6fc23f5-78a7-4129-92ef-b6decff9bf22-kube-api-access-xvf7r\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.388095 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.388058 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ad9f7eb2-071c-4803-96f1-649155c6f4db-agent-certs\") pod \"konnectivity-agent-5bmt4\" (UID: \"ad9f7eb2-071c-4803-96f1-649155c6f4db\") " pod="kube-system/konnectivity-agent-5bmt4" Aug 04 10:27:44.388095 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.388088 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-cnibin\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.388572 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.388126 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-multus-conf-dir\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.388572 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.388196 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cv6w7\" (UniqueName: \"kubernetes.io/projected/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-kube-api-access-cv6w7\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.388572 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.388233 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-run-systemd\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.388572 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.388260 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-var-lib-kubelet\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.388572 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.388291 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7298c7a0-dc2b-4480-aa51-85ba014dcec6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s59fm\" (UID: \"7298c7a0-dc2b-4480-aa51-85ba014dcec6\") " pod="openshift-multus/multus-additional-cni-plugins-s59fm" Aug 04 10:27:44.388787 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.388605 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-etc-sysctl-d\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.388787 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.388652 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-lib-modules\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.388787 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.388684 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-host\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.388787 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.388718 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9p8mx\" (UniqueName: \"kubernetes.io/projected/dfe34fb7-de6d-42c3-b605-5015d4e822c7-kube-api-access-9p8mx\") pod \"iptables-alerter-g4z5z\" (UID: \"dfe34fb7-de6d-42c3-b605-5015d4e822c7\") " pod="openshift-network-operator/iptables-alerter-g4z5z" Aug 04 10:27:44.388787 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.388748 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-systemd-units\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.388994 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.388807 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73272ce5-1238-42c7-b697-29d21e373fc0-original-pull-secret\") pod \"global-pull-secret-syncer-zzlbv\" (UID: \"73272ce5-1238-42c7-b697-29d21e373fc0\") " pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:27:44.388994 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.388914 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzhqr\" (UniqueName: \"kubernetes.io/projected/8c78f70d-8823-4d6c-96f8-c1318ceddd69-kube-api-access-dzhqr\") pod \"aws-ebs-csi-driver-node-5zcvc\" (UID: \"8c78f70d-8823-4d6c-96f8-c1318ceddd69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zcvc" Aug 04 10:27:44.388994 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.388976 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-host-run-multus-certs\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.389133 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.389017 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-run-openvswitch\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.389133 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.389057 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/73272ce5-1238-42c7-b697-29d21e373fc0-dbus\") pod \"global-pull-secret-syncer-zzlbv\" (UID: \"73272ce5-1238-42c7-b697-29d21e373fc0\") " pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:27:44.389133 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.389084 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8c78f70d-8823-4d6c-96f8-c1318ceddd69-socket-dir\") pod \"aws-ebs-csi-driver-node-5zcvc\" (UID: \"8c78f70d-8823-4d6c-96f8-c1318ceddd69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zcvc" Aug 04 10:27:44.389133 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.389127 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8c78f70d-8823-4d6c-96f8-c1318ceddd69-device-dir\") pod \"aws-ebs-csi-driver-node-5zcvc\" (UID: \"8c78f70d-8823-4d6c-96f8-c1318ceddd69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zcvc" Aug 04 10:27:44.389333 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.389159 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7298c7a0-dc2b-4480-aa51-85ba014dcec6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s59fm\" (UID: \"7298c7a0-dc2b-4480-aa51-85ba014dcec6\") " pod="openshift-multus/multus-additional-cni-plugins-s59fm" Aug 04 10:27:44.389333 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.389209 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-os-release\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.389333 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.389243 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-etc-kubernetes\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.389333 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.389293 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-cni-binary-copy\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.389333 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.389318 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-multus-socket-dir-parent\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.389604 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.389348 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-host-var-lib-kubelet\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.389604 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.389405 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-ovnkube-config\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.389604 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.389463 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d6fc23f5-78a7-4129-92ef-b6decff9bf22-etc-tuned\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.390047 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.390023 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6fc23f5-78a7-4129-92ef-b6decff9bf22-tmp\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.390146 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.390075 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ad9f7eb2-071c-4803-96f1-649155c6f4db-konnectivity-ca\") pod \"konnectivity-agent-5bmt4\" (UID: \"ad9f7eb2-071c-4803-96f1-649155c6f4db\") " pod="kube-system/konnectivity-agent-5bmt4" Aug 04 10:27:44.394803 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.394782 2535 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Aug 04 10:27:44.412549 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.412529 2535 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5vxcv" Aug 04 10:27:44.421524 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.421501 2535 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5vxcv" Aug 04 10:27:44.477350 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.477327 2535 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Aug 04 10:27:44.478818 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.478768 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-250.ec2.internal" event={"ID":"b7c39b691124fb3a351388d810da798f","Type":"ContainerStarted","Data":"3376ce2864934d6b3693fac5e488a7dac87364269053c5639b3d98037b6cdb62"} Aug 04 10:27:44.479929 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.479902 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal" event={"ID":"ce61b1a97eb01ff18ec04bf7a4d71f9c","Type":"ContainerStarted","Data":"13b4b701502e797d1da9e7de4f19d774bba21cc6a645d457b62dbd15fe3dd209"} Aug 04 10:27:44.491168 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491145 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-etc-sysctl-d\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.491264 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491179 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-lib-modules\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.491264 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491204 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-host\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.491264 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491221 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9p8mx\" (UniqueName: \"kubernetes.io/projected/dfe34fb7-de6d-42c3-b605-5015d4e822c7-kube-api-access-9p8mx\") pod \"iptables-alerter-g4z5z\" (UID: \"dfe34fb7-de6d-42c3-b605-5015d4e822c7\") " pod="openshift-network-operator/iptables-alerter-g4z5z" Aug 04 10:27:44.491264 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491241 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-systemd-units\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.491457 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491263 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73272ce5-1238-42c7-b697-29d21e373fc0-original-pull-secret\") pod \"global-pull-secret-syncer-zzlbv\" (UID: \"73272ce5-1238-42c7-b697-29d21e373fc0\") " pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:27:44.491457 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491315 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-systemd-units\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.491457 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491327 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-etc-sysctl-d\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.491457 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491325 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-host\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.491457 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491385 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-lib-modules\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.491457 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491429 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzhqr\" (UniqueName: \"kubernetes.io/projected/8c78f70d-8823-4d6c-96f8-c1318ceddd69-kube-api-access-dzhqr\") pod \"aws-ebs-csi-driver-node-5zcvc\" (UID: \"8c78f70d-8823-4d6c-96f8-c1318ceddd69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zcvc" Aug 04 10:27:44.491457 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491458 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-host-run-multus-certs\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.491809 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491476 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-run-openvswitch\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.491809 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491510 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/73272ce5-1238-42c7-b697-29d21e373fc0-dbus\") pod \"global-pull-secret-syncer-zzlbv\" (UID: \"73272ce5-1238-42c7-b697-29d21e373fc0\") " pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:27:44.491809 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:44.491525 2535 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 10:27:44.491809 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491552 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-host-run-multus-certs\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.491809 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491604 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-run-openvswitch\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.491809 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:44.491610 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73272ce5-1238-42c7-b697-29d21e373fc0-original-pull-secret podName:73272ce5-1238-42c7-b697-29d21e373fc0 nodeName:}" failed. No retries permitted until 2026-08-04 10:27:44.991572287 +0000 UTC m=+2.965654812 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/73272ce5-1238-42c7-b697-29d21e373fc0-original-pull-secret") pod "global-pull-secret-syncer-zzlbv" (UID: "73272ce5-1238-42c7-b697-29d21e373fc0") : object "kube-system"/"original-pull-secret" not registered Aug 04 10:27:44.491809 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491635 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8c78f70d-8823-4d6c-96f8-c1318ceddd69-socket-dir\") pod \"aws-ebs-csi-driver-node-5zcvc\" (UID: \"8c78f70d-8823-4d6c-96f8-c1318ceddd69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zcvc" Aug 04 10:27:44.491809 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491667 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8c78f70d-8823-4d6c-96f8-c1318ceddd69-device-dir\") pod \"aws-ebs-csi-driver-node-5zcvc\" (UID: \"8c78f70d-8823-4d6c-96f8-c1318ceddd69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zcvc" Aug 04 10:27:44.491809 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491690 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/73272ce5-1238-42c7-b697-29d21e373fc0-dbus\") pod \"global-pull-secret-syncer-zzlbv\" (UID: \"73272ce5-1238-42c7-b697-29d21e373fc0\") " pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:27:44.491809 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491693 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7298c7a0-dc2b-4480-aa51-85ba014dcec6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s59fm\" (UID: \"7298c7a0-dc2b-4480-aa51-85ba014dcec6\") " pod="openshift-multus/multus-additional-cni-plugins-s59fm" Aug 04 10:27:44.491809 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491720 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-os-release\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.491809 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491745 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-etc-kubernetes\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.491809 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491767 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-cni-binary-copy\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.491809 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491787 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8c78f70d-8823-4d6c-96f8-c1318ceddd69-socket-dir\") pod \"aws-ebs-csi-driver-node-5zcvc\" (UID: \"8c78f70d-8823-4d6c-96f8-c1318ceddd69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zcvc" Aug 04 10:27:44.491809 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491792 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-multus-socket-dir-parent\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.492637 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491813 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-os-release\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.492637 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491815 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-host-var-lib-kubelet\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.492637 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491829 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8c78f70d-8823-4d6c-96f8-c1318ceddd69-device-dir\") pod \"aws-ebs-csi-driver-node-5zcvc\" (UID: \"8c78f70d-8823-4d6c-96f8-c1318ceddd69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zcvc" Aug 04 10:27:44.492637 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491868 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-host-var-lib-kubelet\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.492637 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491881 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-ovnkube-config\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.492637 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491913 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w28sh\" (UniqueName: \"kubernetes.io/projected/b6f16f94-e1b9-4900-a75d-84badd1d9bb7-kube-api-access-w28sh\") pod \"network-check-target-2rc2x\" (UID: \"b6f16f94-e1b9-4900-a75d-84badd1d9bb7\") " pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:27:44.492637 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491948 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-multus-socket-dir-parent\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.492637 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.491993 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d6fc23f5-78a7-4129-92ef-b6decff9bf22-etc-tuned\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.492637 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.492018 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6fc23f5-78a7-4129-92ef-b6decff9bf22-tmp\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.492637 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.492042 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ad9f7eb2-071c-4803-96f1-649155c6f4db-konnectivity-ca\") pod \"konnectivity-agent-5bmt4\" (UID: \"ad9f7eb2-071c-4803-96f1-649155c6f4db\") " pod="kube-system/konnectivity-agent-5bmt4" Aug 04 10:27:44.492637 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.492125 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-etc-kubernetes\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.492637 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.492171 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-ovn-node-metrics-cert\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.492637 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.492200 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8c78f70d-8823-4d6c-96f8-c1318ceddd69-registration-dir\") pod \"aws-ebs-csi-driver-node-5zcvc\" (UID: \"8c78f70d-8823-4d6c-96f8-c1318ceddd69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zcvc" Aug 04 10:27:44.492637 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.492199 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-etc-kubernetes\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.492637 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.492272 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7298c7a0-dc2b-4480-aa51-85ba014dcec6-cnibin\") pod \"multus-additional-cni-plugins-s59fm\" (UID: \"7298c7a0-dc2b-4480-aa51-85ba014dcec6\") " pod="openshift-multus/multus-additional-cni-plugins-s59fm" Aug 04 10:27:44.492637 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.492300 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dfe34fb7-de6d-42c3-b605-5015d4e822c7-iptables-alerter-script\") pod \"iptables-alerter-g4z5z\" (UID: \"dfe34fb7-de6d-42c3-b605-5015d4e822c7\") " pod="openshift-network-operator/iptables-alerter-g4z5z" Aug 04 10:27:44.492637 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.492323 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-host-run-netns\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.493505 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.492368 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-env-overrides\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.493505 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.492379 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-cni-binary-copy\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.493505 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.492395 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/696380db-a93f-48d7-9bb7-108052391f48-tmp-dir\") pod \"node-resolver-s629f\" (UID: \"696380db-a93f-48d7-9bb7-108052391f48\") " pod="openshift-dns/node-resolver-s629f" Aug 04 10:27:44.493505 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.492396 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7298c7a0-dc2b-4480-aa51-85ba014dcec6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s59fm\" (UID: \"7298c7a0-dc2b-4480-aa51-85ba014dcec6\") " pod="openshift-multus/multus-additional-cni-plugins-s59fm" Aug 04 10:27:44.493505 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.492428 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8c78f70d-8823-4d6c-96f8-c1318ceddd69-etc-selinux\") pod \"aws-ebs-csi-driver-node-5zcvc\" (UID: \"8c78f70d-8823-4d6c-96f8-c1318ceddd69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zcvc" Aug 04 10:27:44.493505 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.492465 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7298c7a0-dc2b-4480-aa51-85ba014dcec6-cnibin\") pod \"multus-additional-cni-plugins-s59fm\" (UID: \"7298c7a0-dc2b-4480-aa51-85ba014dcec6\") " pod="openshift-multus/multus-additional-cni-plugins-s59fm" Aug 04 10:27:44.493505 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.492501 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-etc-kubernetes\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.493505 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.492517 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8c78f70d-8823-4d6c-96f8-c1318ceddd69-sys-fs\") pod \"aws-ebs-csi-driver-node-5zcvc\" (UID: \"8c78f70d-8823-4d6c-96f8-c1318ceddd69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zcvc" Aug 04 10:27:44.493505 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.492527 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8c78f70d-8823-4d6c-96f8-c1318ceddd69-registration-dir\") pod \"aws-ebs-csi-driver-node-5zcvc\" (UID: \"8c78f70d-8823-4d6c-96f8-c1318ceddd69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zcvc" Aug 04 10:27:44.493505 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.492538 2535 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Aug 04 10:27:44.493505 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.492602 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8c78f70d-8823-4d6c-96f8-c1318ceddd69-etc-selinux\") pod \"aws-ebs-csi-driver-node-5zcvc\" (UID: \"8c78f70d-8823-4d6c-96f8-c1318ceddd69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zcvc" Aug 04 10:27:44.493505 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.492568 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5dzf\" (UniqueName: \"kubernetes.io/projected/7298c7a0-dc2b-4480-aa51-85ba014dcec6-kube-api-access-h5dzf\") pod \"multus-additional-cni-plugins-s59fm\" (UID: \"7298c7a0-dc2b-4480-aa51-85ba014dcec6\") " pod="openshift-multus/multus-additional-cni-plugins-s59fm" Aug 04 10:27:44.493505 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.492862 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-host-run-k8s-cni-cncf-io\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.493505 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.492893 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-hostroot\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.493505 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.492922 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dfe34fb7-de6d-42c3-b605-5015d4e822c7-host-slash\") pod \"iptables-alerter-g4z5z\" (UID: \"dfe34fb7-de6d-42c3-b605-5015d4e822c7\") " pod="openshift-network-operator/iptables-alerter-g4z5z" Aug 04 10:27:44.493505 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.492922 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-host-run-netns\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.493505 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493100 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7298c7a0-dc2b-4480-aa51-85ba014dcec6-system-cni-dir\") pod \"multus-additional-cni-plugins-s59fm\" (UID: \"7298c7a0-dc2b-4480-aa51-85ba014dcec6\") " pod="openshift-multus/multus-additional-cni-plugins-s59fm" Aug 04 10:27:44.494284 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493134 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7298c7a0-dc2b-4480-aa51-85ba014dcec6-os-release\") pod \"multus-additional-cni-plugins-s59fm\" (UID: \"7298c7a0-dc2b-4480-aa51-85ba014dcec6\") " pod="openshift-multus/multus-additional-cni-plugins-s59fm" Aug 04 10:27:44.494284 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493142 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-hostroot\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.494284 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493151 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-ovnkube-config\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.494284 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493159 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5436bc5b-0740-4ccb-b4fd-857d4023d8e7-metrics-certs\") pod \"network-metrics-daemon-qgpx8\" (UID: \"5436bc5b-0740-4ccb-b4fd-857d4023d8e7\") " pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:27:44.494284 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493189 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-host-kubelet\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.494284 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493104 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dfe34fb7-de6d-42c3-b605-5015d4e822c7-iptables-alerter-script\") pod \"iptables-alerter-g4z5z\" (UID: \"dfe34fb7-de6d-42c3-b605-5015d4e822c7\") " pod="openshift-network-operator/iptables-alerter-g4z5z" Aug 04 10:27:44.494284 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493248 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-host-run-k8s-cni-cncf-io\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.494284 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493256 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-host-kubelet\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.494284 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493256 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7298c7a0-dc2b-4480-aa51-85ba014dcec6-os-release\") pod \"multus-additional-cni-plugins-s59fm\" (UID: \"7298c7a0-dc2b-4480-aa51-85ba014dcec6\") " pod="openshift-multus/multus-additional-cni-plugins-s59fm" Aug 04 10:27:44.494284 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493194 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8c78f70d-8823-4d6c-96f8-c1318ceddd69-sys-fs\") pod \"aws-ebs-csi-driver-node-5zcvc\" (UID: \"8c78f70d-8823-4d6c-96f8-c1318ceddd69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zcvc" Aug 04 10:27:44.494284 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493285 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dfe34fb7-de6d-42c3-b605-5015d4e822c7-host-slash\") pod \"iptables-alerter-g4z5z\" (UID: \"dfe34fb7-de6d-42c3-b605-5015d4e822c7\") " pod="openshift-network-operator/iptables-alerter-g4z5z" Aug 04 10:27:44.494284 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493294 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-env-overrides\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.494284 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:44.493328 2535 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 10:27:44.494284 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493329 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-run-ovn\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.494284 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493332 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7298c7a0-dc2b-4480-aa51-85ba014dcec6-system-cni-dir\") pod \"multus-additional-cni-plugins-s59fm\" (UID: \"7298c7a0-dc2b-4480-aa51-85ba014dcec6\") " pod="openshift-multus/multus-additional-cni-plugins-s59fm" Aug 04 10:27:44.494284 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:44.493430 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5436bc5b-0740-4ccb-b4fd-857d4023d8e7-metrics-certs podName:5436bc5b-0740-4ccb-b4fd-857d4023d8e7 nodeName:}" failed. No retries permitted until 2026-08-04 10:27:44.993414515 +0000 UTC m=+2.967497023 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5436bc5b-0740-4ccb-b4fd-857d4023d8e7-metrics-certs") pod "network-metrics-daemon-qgpx8" (UID: "5436bc5b-0740-4ccb-b4fd-857d4023d8e7") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 10:27:44.494284 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493448 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-log-socket\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.495165 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493470 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-host-cni-netd\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.495165 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493542 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/696380db-a93f-48d7-9bb7-108052391f48-hosts-file\") pod \"node-resolver-s629f\" (UID: \"696380db-a93f-48d7-9bb7-108052391f48\") " pod="openshift-dns/node-resolver-s629f" Aug 04 10:27:44.495165 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493566 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-host-var-lib-cni-multus\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.495165 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493590 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-host-slash\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.495165 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493592 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-run-ovn\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.495165 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493622 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-var-lib-openvswitch\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.495165 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493637 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-host-cni-netd\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.495165 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493644 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-etc-openvswitch\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.495165 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493643 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-host-var-lib-cni-multus\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.495165 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493693 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-log-socket\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.495165 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493715 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9be224ce-6218-4f85-a9f0-a09bf05d4f25-host\") pod \"node-ca-zpdbz\" (UID: \"9be224ce-6218-4f85-a9f0-a09bf05d4f25\") " pod="openshift-image-registry/node-ca-zpdbz" Aug 04 10:27:44.495165 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493704 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-var-lib-openvswitch\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.495165 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493744 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-host-slash\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.495165 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493778 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-etc-modprobe-d\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.495165 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493792 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-etc-openvswitch\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.495165 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493808 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-etc-sysconfig\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.495165 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493845 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-etc-sysctl-conf\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.495165 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493885 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-host-run-ovn-kubernetes\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.496026 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493918 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-etc-modprobe-d\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.496026 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493941 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-host-cni-bin\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.496026 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493967 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.496026 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.493997 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tm8ck\" (UniqueName: \"kubernetes.io/projected/696380db-a93f-48d7-9bb7-108052391f48-kube-api-access-tm8ck\") pod \"node-resolver-s629f\" (UID: \"696380db-a93f-48d7-9bb7-108052391f48\") " pod="openshift-dns/node-resolver-s629f" Aug 04 10:27:44.496026 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.494022 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-etc-sysctl-conf\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.496026 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.494050 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd2p2\" (UniqueName: \"kubernetes.io/projected/9be224ce-6218-4f85-a9f0-a09bf05d4f25-kube-api-access-cd2p2\") pod \"node-ca-zpdbz\" (UID: \"9be224ce-6218-4f85-a9f0-a09bf05d4f25\") " pod="openshift-image-registry/node-ca-zpdbz" Aug 04 10:27:44.496026 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.494084 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-etc-sysconfig\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.496026 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.494077 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-host-run-netns\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.496026 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.494119 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-multus-daemon-config\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.496026 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.494125 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-host-run-netns\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.496026 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.494144 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-node-log\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.496026 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.494171 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-ovnkube-script-lib\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.496026 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.494199 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7298c7a0-dc2b-4480-aa51-85ba014dcec6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s59fm\" (UID: \"7298c7a0-dc2b-4480-aa51-85ba014dcec6\") " pod="openshift-multus/multus-additional-cni-plugins-s59fm" Aug 04 10:27:44.496026 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.494203 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-host-cni-bin\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.496026 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.494246 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-system-cni-dir\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.496026 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.494334 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-host-run-ovn-kubernetes\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.496026 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.494371 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.496963 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.494403 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9be224ce-6218-4f85-a9f0-a09bf05d4f25-serviceca\") pod \"node-ca-zpdbz\" (UID: \"9be224ce-6218-4f85-a9f0-a09bf05d4f25\") " pod="openshift-image-registry/node-ca-zpdbz" Aug 04 10:27:44.496963 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.494443 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-etc-systemd\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.496963 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.494468 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-run\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.496963 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.494667 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-multus-daemon-config\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.496963 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.494736 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-node-log\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.496963 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.494781 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ad9f7eb2-071c-4803-96f1-649155c6f4db-konnectivity-ca\") pod \"konnectivity-agent-5bmt4\" (UID: \"ad9f7eb2-071c-4803-96f1-649155c6f4db\") " pod="kube-system/konnectivity-agent-5bmt4" Aug 04 10:27:44.496963 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.494794 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-etc-systemd\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.496963 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.494842 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-system-cni-dir\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.496963 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.495123 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-run\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.496963 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.495136 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7298c7a0-dc2b-4480-aa51-85ba014dcec6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s59fm\" (UID: \"7298c7a0-dc2b-4480-aa51-85ba014dcec6\") " pod="openshift-multus/multus-additional-cni-plugins-s59fm" Aug 04 10:27:44.496963 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.495231 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/73272ce5-1238-42c7-b697-29d21e373fc0-kubelet-config\") pod \"global-pull-secret-syncer-zzlbv\" (UID: \"73272ce5-1238-42c7-b697-29d21e373fc0\") " pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:27:44.496963 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.495243 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-ovnkube-script-lib\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.496963 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.495259 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7298c7a0-dc2b-4480-aa51-85ba014dcec6-cni-binary-copy\") pod \"multus-additional-cni-plugins-s59fm\" (UID: \"7298c7a0-dc2b-4480-aa51-85ba014dcec6\") " pod="openshift-multus/multus-additional-cni-plugins-s59fm" Aug 04 10:27:44.496963 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.495296 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-multus-cni-dir\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.496963 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.495338 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-sys\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.496963 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.495349 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/73272ce5-1238-42c7-b697-29d21e373fc0-kubelet-config\") pod \"global-pull-secret-syncer-zzlbv\" (UID: \"73272ce5-1238-42c7-b697-29d21e373fc0\") " pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:27:44.496963 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.495365 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8c78f70d-8823-4d6c-96f8-c1318ceddd69-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5zcvc\" (UID: \"8c78f70d-8823-4d6c-96f8-c1318ceddd69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zcvc" Aug 04 10:27:44.496963 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.495446 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-sys\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.497790 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.495424 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-multus-cni-dir\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.497790 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.495409 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8c78f70d-8823-4d6c-96f8-c1318ceddd69-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5zcvc\" (UID: \"8c78f70d-8823-4d6c-96f8-c1318ceddd69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zcvc" Aug 04 10:27:44.497790 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.495506 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-host-var-lib-cni-bin\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.497790 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.495538 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr5nm\" (UniqueName: \"kubernetes.io/projected/5436bc5b-0740-4ccb-b4fd-857d4023d8e7-kube-api-access-zr5nm\") pod \"network-metrics-daemon-qgpx8\" (UID: \"5436bc5b-0740-4ccb-b4fd-857d4023d8e7\") " pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:27:44.497790 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.495564 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qdfx\" (UniqueName: \"kubernetes.io/projected/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-kube-api-access-8qdfx\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.497790 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.495606 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-host-var-lib-cni-bin\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.497790 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.495625 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvf7r\" (UniqueName: \"kubernetes.io/projected/d6fc23f5-78a7-4129-92ef-b6decff9bf22-kube-api-access-xvf7r\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.497790 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.495730 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7298c7a0-dc2b-4480-aa51-85ba014dcec6-cni-binary-copy\") pod \"multus-additional-cni-plugins-s59fm\" (UID: \"7298c7a0-dc2b-4480-aa51-85ba014dcec6\") " pod="openshift-multus/multus-additional-cni-plugins-s59fm" Aug 04 10:27:44.497790 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.495774 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ad9f7eb2-071c-4803-96f1-649155c6f4db-agent-certs\") pod \"konnectivity-agent-5bmt4\" (UID: \"ad9f7eb2-071c-4803-96f1-649155c6f4db\") " pod="kube-system/konnectivity-agent-5bmt4" Aug 04 10:27:44.497790 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.495802 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-cnibin\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.497790 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.495856 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-cnibin\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.497790 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.495932 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-multus-conf-dir\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.497790 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.495982 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv6w7\" (UniqueName: \"kubernetes.io/projected/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-kube-api-access-cv6w7\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.497790 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.496012 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-run-systemd\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.497790 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.496059 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-var-lib-kubelet\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.497790 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.496088 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7298c7a0-dc2b-4480-aa51-85ba014dcec6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s59fm\" (UID: \"7298c7a0-dc2b-4480-aa51-85ba014dcec6\") " pod="openshift-multus/multus-additional-cni-plugins-s59fm" Aug 04 10:27:44.497790 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.496300 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7298c7a0-dc2b-4480-aa51-85ba014dcec6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s59fm\" (UID: \"7298c7a0-dc2b-4480-aa51-85ba014dcec6\") " pod="openshift-multus/multus-additional-cni-plugins-s59fm" Aug 04 10:27:44.497790 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.496378 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-multus-conf-dir\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.498675 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.496403 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-run-systemd\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.498675 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.496539 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6fc23f5-78a7-4129-92ef-b6decff9bf22-var-lib-kubelet\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.498675 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.496583 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d6fc23f5-78a7-4129-92ef-b6decff9bf22-etc-tuned\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.498675 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.497796 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6fc23f5-78a7-4129-92ef-b6decff9bf22-tmp\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.498852 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.498707 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-ovn-node-metrics-cert\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.499000 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.498979 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ad9f7eb2-071c-4803-96f1-649155c6f4db-agent-certs\") pod \"konnectivity-agent-5bmt4\" (UID: \"ad9f7eb2-071c-4803-96f1-649155c6f4db\") " pod="kube-system/konnectivity-agent-5bmt4" Aug 04 10:27:44.502304 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.502286 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9p8mx\" (UniqueName: \"kubernetes.io/projected/dfe34fb7-de6d-42c3-b605-5015d4e822c7-kube-api-access-9p8mx\") pod \"iptables-alerter-g4z5z\" (UID: \"dfe34fb7-de6d-42c3-b605-5015d4e822c7\") " pod="openshift-network-operator/iptables-alerter-g4z5z" Aug 04 10:27:44.503688 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.503618 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzhqr\" (UniqueName: \"kubernetes.io/projected/8c78f70d-8823-4d6c-96f8-c1318ceddd69-kube-api-access-dzhqr\") pod \"aws-ebs-csi-driver-node-5zcvc\" (UID: \"8c78f70d-8823-4d6c-96f8-c1318ceddd69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zcvc" Aug 04 10:27:44.505679 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.505658 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr5nm\" (UniqueName: \"kubernetes.io/projected/5436bc5b-0740-4ccb-b4fd-857d4023d8e7-kube-api-access-zr5nm\") pod \"network-metrics-daemon-qgpx8\" (UID: \"5436bc5b-0740-4ccb-b4fd-857d4023d8e7\") " pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:27:44.505786 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.505759 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvf7r\" (UniqueName: \"kubernetes.io/projected/d6fc23f5-78a7-4129-92ef-b6decff9bf22-kube-api-access-xvf7r\") pod \"tuned-6xm6k\" (UID: \"d6fc23f5-78a7-4129-92ef-b6decff9bf22\") " pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.506506 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.506461 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5dzf\" (UniqueName: \"kubernetes.io/projected/7298c7a0-dc2b-4480-aa51-85ba014dcec6-kube-api-access-h5dzf\") pod \"multus-additional-cni-plugins-s59fm\" (UID: \"7298c7a0-dc2b-4480-aa51-85ba014dcec6\") " pod="openshift-multus/multus-additional-cni-plugins-s59fm" Aug 04 10:27:44.507731 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.507704 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qdfx\" (UniqueName: \"kubernetes.io/projected/8c588e91-5d0d-48d2-a8aa-e0231bfbc654-kube-api-access-8qdfx\") pod \"ovnkube-node-lcxhd\" (UID: \"8c588e91-5d0d-48d2-a8aa-e0231bfbc654\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.507795 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.507779 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cv6w7\" (UniqueName: \"kubernetes.io/projected/c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9-kube-api-access-cv6w7\") pod \"multus-6xqqk\" (UID: \"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9\") " pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.542871 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.542849 2535 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 10:27:44.596644 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.596612 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w28sh\" (UniqueName: \"kubernetes.io/projected/b6f16f94-e1b9-4900-a75d-84badd1d9bb7-kube-api-access-w28sh\") pod \"network-check-target-2rc2x\" (UID: \"b6f16f94-e1b9-4900-a75d-84badd1d9bb7\") " pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:27:44.596821 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.596661 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/696380db-a93f-48d7-9bb7-108052391f48-tmp-dir\") pod \"node-resolver-s629f\" (UID: \"696380db-a93f-48d7-9bb7-108052391f48\") " pod="openshift-dns/node-resolver-s629f" Aug 04 10:27:44.596821 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.596708 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/696380db-a93f-48d7-9bb7-108052391f48-hosts-file\") pod \"node-resolver-s629f\" (UID: \"696380db-a93f-48d7-9bb7-108052391f48\") " pod="openshift-dns/node-resolver-s629f" Aug 04 10:27:44.596821 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.596733 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9be224ce-6218-4f85-a9f0-a09bf05d4f25-host\") pod \"node-ca-zpdbz\" (UID: \"9be224ce-6218-4f85-a9f0-a09bf05d4f25\") " pod="openshift-image-registry/node-ca-zpdbz" Aug 04 10:27:44.596821 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.596762 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm8ck\" (UniqueName: \"kubernetes.io/projected/696380db-a93f-48d7-9bb7-108052391f48-kube-api-access-tm8ck\") pod \"node-resolver-s629f\" (UID: \"696380db-a93f-48d7-9bb7-108052391f48\") " pod="openshift-dns/node-resolver-s629f" Aug 04 10:27:44.596821 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.596807 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd2p2\" (UniqueName: \"kubernetes.io/projected/9be224ce-6218-4f85-a9f0-a09bf05d4f25-kube-api-access-cd2p2\") pod \"node-ca-zpdbz\" (UID: \"9be224ce-6218-4f85-a9f0-a09bf05d4f25\") " pod="openshift-image-registry/node-ca-zpdbz" Aug 04 10:27:44.597087 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.596834 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/696380db-a93f-48d7-9bb7-108052391f48-hosts-file\") pod \"node-resolver-s629f\" (UID: \"696380db-a93f-48d7-9bb7-108052391f48\") " pod="openshift-dns/node-resolver-s629f" Aug 04 10:27:44.597087 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.596853 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9be224ce-6218-4f85-a9f0-a09bf05d4f25-serviceca\") pod \"node-ca-zpdbz\" (UID: \"9be224ce-6218-4f85-a9f0-a09bf05d4f25\") " pod="openshift-image-registry/node-ca-zpdbz" Aug 04 10:27:44.597087 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.596941 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9be224ce-6218-4f85-a9f0-a09bf05d4f25-host\") pod \"node-ca-zpdbz\" (UID: \"9be224ce-6218-4f85-a9f0-a09bf05d4f25\") " pod="openshift-image-registry/node-ca-zpdbz" Aug 04 10:27:44.597294 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.597270 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/696380db-a93f-48d7-9bb7-108052391f48-tmp-dir\") pod \"node-resolver-s629f\" (UID: \"696380db-a93f-48d7-9bb7-108052391f48\") " pod="openshift-dns/node-resolver-s629f" Aug 04 10:27:44.597398 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.597379 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9be224ce-6218-4f85-a9f0-a09bf05d4f25-serviceca\") pod \"node-ca-zpdbz\" (UID: \"9be224ce-6218-4f85-a9f0-a09bf05d4f25\") " pod="openshift-image-registry/node-ca-zpdbz" Aug 04 10:27:44.603889 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:44.603869 2535 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 10:27:44.603984 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:44.603893 2535 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 10:27:44.603984 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:44.603912 2535 projected.go:196] Error preparing data for projected volume kube-api-access-w28sh for pod openshift-network-diagnostics/network-check-target-2rc2x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 10:27:44.603984 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:44.603979 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b6f16f94-e1b9-4900-a75d-84badd1d9bb7-kube-api-access-w28sh podName:b6f16f94-e1b9-4900-a75d-84badd1d9bb7 nodeName:}" failed. No retries permitted until 2026-08-04 10:27:45.103961715 +0000 UTC m=+3.078044220 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-w28sh" (UniqueName: "kubernetes.io/projected/b6f16f94-e1b9-4900-a75d-84badd1d9bb7-kube-api-access-w28sh") pod "network-check-target-2rc2x" (UID: "b6f16f94-e1b9-4900-a75d-84badd1d9bb7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 10:27:44.606143 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.606091 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd2p2\" (UniqueName: \"kubernetes.io/projected/9be224ce-6218-4f85-a9f0-a09bf05d4f25-kube-api-access-cd2p2\") pod \"node-ca-zpdbz\" (UID: \"9be224ce-6218-4f85-a9f0-a09bf05d4f25\") " pod="openshift-image-registry/node-ca-zpdbz" Aug 04 10:27:44.606227 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.606161 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tm8ck\" (UniqueName: \"kubernetes.io/projected/696380db-a93f-48d7-9bb7-108052391f48-kube-api-access-tm8ck\") pod \"node-resolver-s629f\" (UID: \"696380db-a93f-48d7-9bb7-108052391f48\") " pod="openshift-dns/node-resolver-s629f" Aug 04 10:27:44.679057 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.679024 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5bmt4" Aug 04 10:27:44.687900 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.687872 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6xqqk" Aug 04 10:27:44.696528 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.696506 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-s59fm" Aug 04 10:27:44.704628 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.704605 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-g4z5z" Aug 04 10:27:44.713375 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.713359 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:27:44.716817 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.716799 2535 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 10:27:44.720472 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.720455 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" Aug 04 10:27:44.728031 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.728010 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zcvc" Aug 04 10:27:44.735384 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.735367 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s629f" Aug 04 10:27:44.747814 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:44.747798 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zpdbz" Aug 04 10:27:45.000648 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:45.000575 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73272ce5-1238-42c7-b697-29d21e373fc0-original-pull-secret\") pod \"global-pull-secret-syncer-zzlbv\" (UID: \"73272ce5-1238-42c7-b697-29d21e373fc0\") " pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:27:45.000784 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:45.000659 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5436bc5b-0740-4ccb-b4fd-857d4023d8e7-metrics-certs\") pod \"network-metrics-daemon-qgpx8\" (UID: \"5436bc5b-0740-4ccb-b4fd-857d4023d8e7\") " pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:27:45.000784 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:45.000732 2535 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 10:27:45.000899 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:45.000785 2535 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 10:27:45.000899 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:45.000805 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73272ce5-1238-42c7-b697-29d21e373fc0-original-pull-secret podName:73272ce5-1238-42c7-b697-29d21e373fc0 nodeName:}" failed. No retries permitted until 2026-08-04 10:27:46.000781477 +0000 UTC m=+3.974864007 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/73272ce5-1238-42c7-b697-29d21e373fc0-original-pull-secret") pod "global-pull-secret-syncer-zzlbv" (UID: "73272ce5-1238-42c7-b697-29d21e373fc0") : object "kube-system"/"original-pull-secret" not registered Aug 04 10:27:45.000899 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:45.000845 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5436bc5b-0740-4ccb-b4fd-857d4023d8e7-metrics-certs podName:5436bc5b-0740-4ccb-b4fd-857d4023d8e7 nodeName:}" failed. No retries permitted until 2026-08-04 10:27:46.000828188 +0000 UTC m=+3.974910701 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5436bc5b-0740-4ccb-b4fd-857d4023d8e7-metrics-certs") pod "network-metrics-daemon-qgpx8" (UID: "5436bc5b-0740-4ccb-b4fd-857d4023d8e7") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 10:27:45.203075 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:45.203033 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w28sh\" (UniqueName: \"kubernetes.io/projected/b6f16f94-e1b9-4900-a75d-84badd1d9bb7-kube-api-access-w28sh\") pod \"network-check-target-2rc2x\" (UID: \"b6f16f94-e1b9-4900-a75d-84badd1d9bb7\") " pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:27:45.203227 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:45.203204 2535 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 10:27:45.203227 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:45.203225 2535 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 10:27:45.203296 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:45.203236 2535 projected.go:196] Error preparing data for projected volume kube-api-access-w28sh for pod openshift-network-diagnostics/network-check-target-2rc2x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 10:27:45.203344 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:45.203338 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b6f16f94-e1b9-4900-a75d-84badd1d9bb7-kube-api-access-w28sh podName:b6f16f94-e1b9-4900-a75d-84badd1d9bb7 nodeName:}" failed. No retries permitted until 2026-08-04 10:27:46.203312958 +0000 UTC m=+4.177395463 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-w28sh" (UniqueName: "kubernetes.io/projected/b6f16f94-e1b9-4900-a75d-84badd1d9bb7-kube-api-access-w28sh") pod "network-check-target-2rc2x" (UID: "b6f16f94-e1b9-4900-a75d-84badd1d9bb7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 10:27:45.337941 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:45.337909 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad9f7eb2_071c_4803_96f1_649155c6f4db.slice/crio-98a3de04b1912bb9ef2fd8077d7f1cbef5712baa3585ddae3c3442c59097d715 WatchSource:0}: Error finding container 98a3de04b1912bb9ef2fd8077d7f1cbef5712baa3585ddae3c3442c59097d715: Status 404 returned error can't find the container with id 98a3de04b1912bb9ef2fd8077d7f1cbef5712baa3585ddae3c3442c59097d715 Aug 04 10:27:45.339185 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:45.339163 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c78f70d_8823_4d6c_96f8_c1318ceddd69.slice/crio-3e533d43a315f0fb06fa3712f22d5d8b791ddef52c11b1d56ffe0d5e5b55a267 WatchSource:0}: Error finding container 3e533d43a315f0fb06fa3712f22d5d8b791ddef52c11b1d56ffe0d5e5b55a267: Status 404 returned error can't find the container with id 3e533d43a315f0fb06fa3712f22d5d8b791ddef52c11b1d56ffe0d5e5b55a267 Aug 04 10:27:45.340808 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:45.340714 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9be224ce_6218_4f85_a9f0_a09bf05d4f25.slice/crio-abc776c4f843aebed8f4b30f40c239e247c7e58e9b50a34903e700f7e7cd5f50 WatchSource:0}: Error finding container abc776c4f843aebed8f4b30f40c239e247c7e58e9b50a34903e700f7e7cd5f50: Status 404 returned error can't find the container with id abc776c4f843aebed8f4b30f40c239e247c7e58e9b50a34903e700f7e7cd5f50 Aug 04 10:27:45.342285 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:45.342227 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod696380db_a93f_48d7_9bb7_108052391f48.slice/crio-224f06553e3ca226043dc01ac9a0479b03e5bedb1eeba9b656c37366053e30cd WatchSource:0}: Error finding container 224f06553e3ca226043dc01ac9a0479b03e5bedb1eeba9b656c37366053e30cd: Status 404 returned error can't find the container with id 224f06553e3ca226043dc01ac9a0479b03e5bedb1eeba9b656c37366053e30cd Aug 04 10:27:45.345032 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:45.345010 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c588e91_5d0d_48d2_a8aa_e0231bfbc654.slice/crio-8e46013923076cbabfeabaaf8d8d67ca0f1fbf7da39a057a06e07bd8a37f6fb5 WatchSource:0}: Error finding container 8e46013923076cbabfeabaaf8d8d67ca0f1fbf7da39a057a06e07bd8a37f6fb5: Status 404 returned error can't find the container with id 8e46013923076cbabfeabaaf8d8d67ca0f1fbf7da39a057a06e07bd8a37f6fb5 Aug 04 10:27:45.346645 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:45.346304 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6fc23f5_78a7_4129_92ef_b6decff9bf22.slice/crio-4e1d498c8e918e0b07c4fc52a034529ffc33dc16f88bbb9298ee254b41895783 WatchSource:0}: Error finding container 4e1d498c8e918e0b07c4fc52a034529ffc33dc16f88bbb9298ee254b41895783: Status 404 returned error can't find the container with id 4e1d498c8e918e0b07c4fc52a034529ffc33dc16f88bbb9298ee254b41895783 Aug 04 10:27:45.347197 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:45.347176 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7298c7a0_dc2b_4480_aa51_85ba014dcec6.slice/crio-102c9d15efc2796815df180a5a4e189571161416e8d19214f6b4cc8cf4458d03 WatchSource:0}: Error finding container 102c9d15efc2796815df180a5a4e189571161416e8d19214f6b4cc8cf4458d03: Status 404 returned error can't find the container with id 102c9d15efc2796815df180a5a4e189571161416e8d19214f6b4cc8cf4458d03 Aug 04 10:27:45.348655 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:45.348582 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfe34fb7_de6d_42c3_b605_5015d4e822c7.slice/crio-f5802a1020ca9c8bf31051a99f0ecfa3519285a9783c0433544ee4f956f58464 WatchSource:0}: Error finding container f5802a1020ca9c8bf31051a99f0ecfa3519285a9783c0433544ee4f956f58464: Status 404 returned error can't find the container with id f5802a1020ca9c8bf31051a99f0ecfa3519285a9783c0433544ee4f956f58464 Aug 04 10:27:45.350171 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:27:45.349960 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc581a5bc_9508_4bc7_bc61_6bbcc0bf0af9.slice/crio-8a67971c9408b0c37be71479309c99ee0dc2eac58fa98fc809d5f0e1053f75e3 WatchSource:0}: Error finding container 8a67971c9408b0c37be71479309c99ee0dc2eac58fa98fc809d5f0e1053f75e3: Status 404 returned error can't find the container with id 8a67971c9408b0c37be71479309c99ee0dc2eac58fa98fc809d5f0e1053f75e3 Aug 04 10:27:45.424314 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:45.424164 2535 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 10:22:44 +0000 UTC" deadline="2028-02-12 05:44:11.951818767 +0000 UTC" Aug 04 10:27:45.424702 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:45.424317 2535 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13363h16m26.52750752s" Aug 04 10:27:45.483134 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:45.483106 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s629f" event={"ID":"696380db-a93f-48d7-9bb7-108052391f48","Type":"ContainerStarted","Data":"224f06553e3ca226043dc01ac9a0479b03e5bedb1eeba9b656c37366053e30cd"} Aug 04 10:27:45.484119 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:45.484082 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zpdbz" event={"ID":"9be224ce-6218-4f85-a9f0-a09bf05d4f25","Type":"ContainerStarted","Data":"abc776c4f843aebed8f4b30f40c239e247c7e58e9b50a34903e700f7e7cd5f50"} Aug 04 10:27:45.485288 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:45.485237 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5bmt4" event={"ID":"ad9f7eb2-071c-4803-96f1-649155c6f4db","Type":"ContainerStarted","Data":"98a3de04b1912bb9ef2fd8077d7f1cbef5712baa3585ddae3c3442c59097d715"} Aug 04 10:27:45.486326 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:45.486306 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-g4z5z" event={"ID":"dfe34fb7-de6d-42c3-b605-5015d4e822c7","Type":"ContainerStarted","Data":"f5802a1020ca9c8bf31051a99f0ecfa3519285a9783c0433544ee4f956f58464"} Aug 04 10:27:45.487361 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:45.487343 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s59fm" event={"ID":"7298c7a0-dc2b-4480-aa51-85ba014dcec6","Type":"ContainerStarted","Data":"102c9d15efc2796815df180a5a4e189571161416e8d19214f6b4cc8cf4458d03"} Aug 04 10:27:45.488300 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:45.488280 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" event={"ID":"d6fc23f5-78a7-4129-92ef-b6decff9bf22","Type":"ContainerStarted","Data":"4e1d498c8e918e0b07c4fc52a034529ffc33dc16f88bbb9298ee254b41895783"} Aug 04 10:27:45.489250 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:45.489231 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zcvc" event={"ID":"8c78f70d-8823-4d6c-96f8-c1318ceddd69","Type":"ContainerStarted","Data":"3e533d43a315f0fb06fa3712f22d5d8b791ddef52c11b1d56ffe0d5e5b55a267"} Aug 04 10:27:45.492160 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:45.492131 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-250.ec2.internal" event={"ID":"b7c39b691124fb3a351388d810da798f","Type":"ContainerStarted","Data":"dd99f3c22a3ab187f8ac621e0d04a09fce1a1db2701a5c5b2c70d76f87934df4"} Aug 04 10:27:45.493342 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:45.493318 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6xqqk" event={"ID":"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9","Type":"ContainerStarted","Data":"8a67971c9408b0c37be71479309c99ee0dc2eac58fa98fc809d5f0e1053f75e3"} Aug 04 10:27:45.497750 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:45.497729 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" event={"ID":"8c588e91-5d0d-48d2-a8aa-e0231bfbc654","Type":"ContainerStarted","Data":"8e46013923076cbabfeabaaf8d8d67ca0f1fbf7da39a057a06e07bd8a37f6fb5"} Aug 04 10:27:45.508381 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:45.508337 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-250.ec2.internal" podStartSLOduration=2.5083222899999997 podStartE2EDuration="2.50832229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 10:27:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:27:45.507893123 +0000 UTC m=+3.481975650" watchObservedRunningTime="2026-08-04 10:27:45.50832229 +0000 UTC m=+3.482404818" Aug 04 10:27:46.010585 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:46.010466 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73272ce5-1238-42c7-b697-29d21e373fc0-original-pull-secret\") pod \"global-pull-secret-syncer-zzlbv\" (UID: \"73272ce5-1238-42c7-b697-29d21e373fc0\") " pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:27:46.010733 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:46.010595 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5436bc5b-0740-4ccb-b4fd-857d4023d8e7-metrics-certs\") pod \"network-metrics-daemon-qgpx8\" (UID: \"5436bc5b-0740-4ccb-b4fd-857d4023d8e7\") " pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:27:46.010791 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:46.010745 2535 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 10:27:46.010849 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:46.010806 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5436bc5b-0740-4ccb-b4fd-857d4023d8e7-metrics-certs podName:5436bc5b-0740-4ccb-b4fd-857d4023d8e7 nodeName:}" failed. No retries permitted until 2026-08-04 10:27:48.010788942 +0000 UTC m=+5.984871453 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5436bc5b-0740-4ccb-b4fd-857d4023d8e7-metrics-certs") pod "network-metrics-daemon-qgpx8" (UID: "5436bc5b-0740-4ccb-b4fd-857d4023d8e7") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 10:27:46.011200 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:46.011184 2535 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 10:27:46.011347 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:46.011235 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73272ce5-1238-42c7-b697-29d21e373fc0-original-pull-secret podName:73272ce5-1238-42c7-b697-29d21e373fc0 nodeName:}" failed. No retries permitted until 2026-08-04 10:27:48.011220321 +0000 UTC m=+5.985302831 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/73272ce5-1238-42c7-b697-29d21e373fc0-original-pull-secret") pod "global-pull-secret-syncer-zzlbv" (UID: "73272ce5-1238-42c7-b697-29d21e373fc0") : object "kube-system"/"original-pull-secret" not registered Aug 04 10:27:46.212634 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:46.212597 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w28sh\" (UniqueName: \"kubernetes.io/projected/b6f16f94-e1b9-4900-a75d-84badd1d9bb7-kube-api-access-w28sh\") pod \"network-check-target-2rc2x\" (UID: \"b6f16f94-e1b9-4900-a75d-84badd1d9bb7\") " pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:27:46.212812 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:46.212791 2535 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 10:27:46.212886 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:46.212820 2535 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 10:27:46.212886 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:46.212834 2535 projected.go:196] Error preparing data for projected volume kube-api-access-w28sh for pod openshift-network-diagnostics/network-check-target-2rc2x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 10:27:46.212988 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:46.212906 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b6f16f94-e1b9-4900-a75d-84badd1d9bb7-kube-api-access-w28sh podName:b6f16f94-e1b9-4900-a75d-84badd1d9bb7 nodeName:}" failed. No retries permitted until 2026-08-04 10:27:48.212885933 +0000 UTC m=+6.186968442 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-w28sh" (UniqueName: "kubernetes.io/projected/b6f16f94-e1b9-4900-a75d-84badd1d9bb7-kube-api-access-w28sh") pod "network-check-target-2rc2x" (UID: "b6f16f94-e1b9-4900-a75d-84badd1d9bb7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 10:27:46.424814 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:46.424749 2535 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 10:22:44 +0000 UTC" deadline="2028-04-28 04:43:14.708497436 +0000 UTC" Aug 04 10:27:46.424814 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:46.424812 2535 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15186h15m28.283689277s" Aug 04 10:27:46.474669 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:46.474607 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:27:46.474844 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:46.474740 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgpx8" podUID="5436bc5b-0740-4ccb-b4fd-857d4023d8e7" Aug 04 10:27:46.475138 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:46.475119 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:27:46.475235 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:46.475214 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zzlbv" podUID="73272ce5-1238-42c7-b697-29d21e373fc0" Aug 04 10:27:46.475299 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:46.475279 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:27:46.475382 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:46.475350 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rc2x" podUID="b6f16f94-e1b9-4900-a75d-84badd1d9bb7" Aug 04 10:27:46.512953 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:46.512909 2535 generic.go:356] "Generic (PLEG): container finished" podID="ce61b1a97eb01ff18ec04bf7a4d71f9c" containerID="5b24c2a8f5cc4711c63b612d294820dac781c5f5aa0622553ae5f9a78952ce91" exitCode=0 Aug 04 10:27:46.513178 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:46.513034 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal" event={"ID":"ce61b1a97eb01ff18ec04bf7a4d71f9c","Type":"ContainerDied","Data":"5b24c2a8f5cc4711c63b612d294820dac781c5f5aa0622553ae5f9a78952ce91"} Aug 04 10:27:47.529989 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:47.529954 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal" event={"ID":"ce61b1a97eb01ff18ec04bf7a4d71f9c","Type":"ContainerStarted","Data":"2a0818ee1a3ce6a943876a21a131ac9b1ff999bb6c5d0e4c5a0c866c46389d06"} Aug 04 10:27:48.027646 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:48.027542 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73272ce5-1238-42c7-b697-29d21e373fc0-original-pull-secret\") pod \"global-pull-secret-syncer-zzlbv\" (UID: \"73272ce5-1238-42c7-b697-29d21e373fc0\") " pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:27:48.027646 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:48.027622 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5436bc5b-0740-4ccb-b4fd-857d4023d8e7-metrics-certs\") pod \"network-metrics-daemon-qgpx8\" (UID: \"5436bc5b-0740-4ccb-b4fd-857d4023d8e7\") " pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:27:48.027865 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:48.027775 2535 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 10:27:48.027920 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:48.027872 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5436bc5b-0740-4ccb-b4fd-857d4023d8e7-metrics-certs podName:5436bc5b-0740-4ccb-b4fd-857d4023d8e7 nodeName:}" failed. No retries permitted until 2026-08-04 10:27:52.02784945 +0000 UTC m=+10.001931974 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5436bc5b-0740-4ccb-b4fd-857d4023d8e7-metrics-certs") pod "network-metrics-daemon-qgpx8" (UID: "5436bc5b-0740-4ccb-b4fd-857d4023d8e7") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 10:27:48.028311 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:48.028292 2535 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 10:27:48.028422 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:48.028345 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73272ce5-1238-42c7-b697-29d21e373fc0-original-pull-secret podName:73272ce5-1238-42c7-b697-29d21e373fc0 nodeName:}" failed. No retries permitted until 2026-08-04 10:27:52.028331045 +0000 UTC m=+10.002413566 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/73272ce5-1238-42c7-b697-29d21e373fc0-original-pull-secret") pod "global-pull-secret-syncer-zzlbv" (UID: "73272ce5-1238-42c7-b697-29d21e373fc0") : object "kube-system"/"original-pull-secret" not registered Aug 04 10:27:48.230003 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:48.229369 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w28sh\" (UniqueName: \"kubernetes.io/projected/b6f16f94-e1b9-4900-a75d-84badd1d9bb7-kube-api-access-w28sh\") pod \"network-check-target-2rc2x\" (UID: \"b6f16f94-e1b9-4900-a75d-84badd1d9bb7\") " pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:27:48.230003 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:48.229575 2535 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 10:27:48.230003 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:48.229603 2535 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 10:27:48.230003 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:48.229616 2535 projected.go:196] Error preparing data for projected volume kube-api-access-w28sh for pod openshift-network-diagnostics/network-check-target-2rc2x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 10:27:48.230003 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:48.229676 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b6f16f94-e1b9-4900-a75d-84badd1d9bb7-kube-api-access-w28sh podName:b6f16f94-e1b9-4900-a75d-84badd1d9bb7 nodeName:}" failed. No retries permitted until 2026-08-04 10:27:52.229657586 +0000 UTC m=+10.203740094 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-w28sh" (UniqueName: "kubernetes.io/projected/b6f16f94-e1b9-4900-a75d-84badd1d9bb7-kube-api-access-w28sh") pod "network-check-target-2rc2x" (UID: "b6f16f94-e1b9-4900-a75d-84badd1d9bb7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 10:27:48.474679 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:48.474581 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:27:48.474848 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:48.474744 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zzlbv" podUID="73272ce5-1238-42c7-b697-29d21e373fc0" Aug 04 10:27:48.474848 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:48.474759 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:27:48.474966 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:48.474854 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rc2x" podUID="b6f16f94-e1b9-4900-a75d-84badd1d9bb7" Aug 04 10:27:48.474966 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:48.474910 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:27:48.475061 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:48.474990 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgpx8" podUID="5436bc5b-0740-4ccb-b4fd-857d4023d8e7" Aug 04 10:27:50.476650 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:50.476445 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:27:50.476650 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:50.476458 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:27:50.476650 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:50.476596 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rc2x" podUID="b6f16f94-e1b9-4900-a75d-84badd1d9bb7" Aug 04 10:27:50.476650 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:50.476608 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:27:50.477199 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:50.476710 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgpx8" podUID="5436bc5b-0740-4ccb-b4fd-857d4023d8e7" Aug 04 10:27:50.477199 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:50.476824 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zzlbv" podUID="73272ce5-1238-42c7-b697-29d21e373fc0" Aug 04 10:27:52.060514 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:52.060219 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5436bc5b-0740-4ccb-b4fd-857d4023d8e7-metrics-certs\") pod \"network-metrics-daemon-qgpx8\" (UID: \"5436bc5b-0740-4ccb-b4fd-857d4023d8e7\") " pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:27:52.060514 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:52.060293 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73272ce5-1238-42c7-b697-29d21e373fc0-original-pull-secret\") pod \"global-pull-secret-syncer-zzlbv\" (UID: \"73272ce5-1238-42c7-b697-29d21e373fc0\") " pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:27:52.060514 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:52.060347 2535 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 10:27:52.060514 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:52.060407 2535 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 10:27:52.060514 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:52.060412 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5436bc5b-0740-4ccb-b4fd-857d4023d8e7-metrics-certs podName:5436bc5b-0740-4ccb-b4fd-857d4023d8e7 nodeName:}" failed. No retries permitted until 2026-08-04 10:28:00.060392801 +0000 UTC m=+18.034475312 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5436bc5b-0740-4ccb-b4fd-857d4023d8e7-metrics-certs") pod "network-metrics-daemon-qgpx8" (UID: "5436bc5b-0740-4ccb-b4fd-857d4023d8e7") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 10:27:52.060514 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:52.060468 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73272ce5-1238-42c7-b697-29d21e373fc0-original-pull-secret podName:73272ce5-1238-42c7-b697-29d21e373fc0 nodeName:}" failed. No retries permitted until 2026-08-04 10:28:00.060453227 +0000 UTC m=+18.034535736 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/73272ce5-1238-42c7-b697-29d21e373fc0-original-pull-secret") pod "global-pull-secret-syncer-zzlbv" (UID: "73272ce5-1238-42c7-b697-29d21e373fc0") : object "kube-system"/"original-pull-secret" not registered Aug 04 10:27:52.263274 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:52.262714 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w28sh\" (UniqueName: \"kubernetes.io/projected/b6f16f94-e1b9-4900-a75d-84badd1d9bb7-kube-api-access-w28sh\") pod \"network-check-target-2rc2x\" (UID: \"b6f16f94-e1b9-4900-a75d-84badd1d9bb7\") " pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:27:52.263274 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:52.262864 2535 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 10:27:52.263274 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:52.262882 2535 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 10:27:52.263274 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:52.262894 2535 projected.go:196] Error preparing data for projected volume kube-api-access-w28sh for pod openshift-network-diagnostics/network-check-target-2rc2x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 10:27:52.263274 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:52.262949 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b6f16f94-e1b9-4900-a75d-84badd1d9bb7-kube-api-access-w28sh podName:b6f16f94-e1b9-4900-a75d-84badd1d9bb7 nodeName:}" failed. No retries permitted until 2026-08-04 10:28:00.262930683 +0000 UTC m=+18.237013193 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-w28sh" (UniqueName: "kubernetes.io/projected/b6f16f94-e1b9-4900-a75d-84badd1d9bb7-kube-api-access-w28sh") pod "network-check-target-2rc2x" (UID: "b6f16f94-e1b9-4900-a75d-84badd1d9bb7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 10:27:52.476048 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:52.475301 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:27:52.476048 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:52.475412 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rc2x" podUID="b6f16f94-e1b9-4900-a75d-84badd1d9bb7" Aug 04 10:27:52.476048 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:52.475768 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:27:52.476048 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:52.475865 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgpx8" podUID="5436bc5b-0740-4ccb-b4fd-857d4023d8e7" Aug 04 10:27:52.476048 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:52.475906 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:27:52.476048 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:52.475973 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zzlbv" podUID="73272ce5-1238-42c7-b697-29d21e373fc0" Aug 04 10:27:54.474517 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:54.474447 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:27:54.474954 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:54.474473 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:27:54.474954 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:54.474475 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:27:54.474954 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:54.474660 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zzlbv" podUID="73272ce5-1238-42c7-b697-29d21e373fc0" Aug 04 10:27:54.474954 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:54.474763 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rc2x" podUID="b6f16f94-e1b9-4900-a75d-84badd1d9bb7" Aug 04 10:27:54.474954 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:54.474848 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgpx8" podUID="5436bc5b-0740-4ccb-b4fd-857d4023d8e7" Aug 04 10:27:56.474123 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:56.474062 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:27:56.474123 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:56.474114 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:27:56.474677 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:56.474062 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:27:56.474677 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:56.474207 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgpx8" podUID="5436bc5b-0740-4ccb-b4fd-857d4023d8e7" Aug 04 10:27:56.474677 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:56.474316 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rc2x" podUID="b6f16f94-e1b9-4900-a75d-84badd1d9bb7" Aug 04 10:27:56.474677 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:56.474398 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zzlbv" podUID="73272ce5-1238-42c7-b697-29d21e373fc0" Aug 04 10:27:58.474070 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:58.474039 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:27:58.474070 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:58.474055 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:27:58.474614 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:58.474165 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rc2x" podUID="b6f16f94-e1b9-4900-a75d-84badd1d9bb7" Aug 04 10:27:58.474614 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:27:58.474263 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:27:58.474614 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:58.474256 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgpx8" podUID="5436bc5b-0740-4ccb-b4fd-857d4023d8e7" Aug 04 10:27:58.474614 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:27:58.474357 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zzlbv" podUID="73272ce5-1238-42c7-b697-29d21e373fc0" Aug 04 10:28:00.121818 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:00.121783 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5436bc5b-0740-4ccb-b4fd-857d4023d8e7-metrics-certs\") pod \"network-metrics-daemon-qgpx8\" (UID: \"5436bc5b-0740-4ccb-b4fd-857d4023d8e7\") " pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:28:00.122312 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:00.121840 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73272ce5-1238-42c7-b697-29d21e373fc0-original-pull-secret\") pod \"global-pull-secret-syncer-zzlbv\" (UID: \"73272ce5-1238-42c7-b697-29d21e373fc0\") " pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:28:00.122312 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:00.121933 2535 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 10:28:00.122312 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:00.121992 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5436bc5b-0740-4ccb-b4fd-857d4023d8e7-metrics-certs podName:5436bc5b-0740-4ccb-b4fd-857d4023d8e7 nodeName:}" failed. No retries permitted until 2026-08-04 10:28:16.121978592 +0000 UTC m=+34.096061100 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5436bc5b-0740-4ccb-b4fd-857d4023d8e7-metrics-certs") pod "network-metrics-daemon-qgpx8" (UID: "5436bc5b-0740-4ccb-b4fd-857d4023d8e7") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 10:28:00.122312 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:00.121933 2535 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 10:28:00.122312 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:00.122063 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73272ce5-1238-42c7-b697-29d21e373fc0-original-pull-secret podName:73272ce5-1238-42c7-b697-29d21e373fc0 nodeName:}" failed. No retries permitted until 2026-08-04 10:28:16.122051699 +0000 UTC m=+34.096134204 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/73272ce5-1238-42c7-b697-29d21e373fc0-original-pull-secret") pod "global-pull-secret-syncer-zzlbv" (UID: "73272ce5-1238-42c7-b697-29d21e373fc0") : object "kube-system"/"original-pull-secret" not registered Aug 04 10:28:00.322901 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:00.322865 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w28sh\" (UniqueName: \"kubernetes.io/projected/b6f16f94-e1b9-4900-a75d-84badd1d9bb7-kube-api-access-w28sh\") pod \"network-check-target-2rc2x\" (UID: \"b6f16f94-e1b9-4900-a75d-84badd1d9bb7\") " pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:28:00.323056 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:00.323034 2535 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 10:28:00.323114 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:00.323067 2535 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 10:28:00.323114 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:00.323082 2535 projected.go:196] Error preparing data for projected volume kube-api-access-w28sh for pod openshift-network-diagnostics/network-check-target-2rc2x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 10:28:00.323193 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:00.323145 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b6f16f94-e1b9-4900-a75d-84badd1d9bb7-kube-api-access-w28sh podName:b6f16f94-e1b9-4900-a75d-84badd1d9bb7 nodeName:}" failed. No retries permitted until 2026-08-04 10:28:16.323127412 +0000 UTC m=+34.297209920 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-w28sh" (UniqueName: "kubernetes.io/projected/b6f16f94-e1b9-4900-a75d-84badd1d9bb7-kube-api-access-w28sh") pod "network-check-target-2rc2x" (UID: "b6f16f94-e1b9-4900-a75d-84badd1d9bb7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 10:28:00.474012 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:00.473926 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:28:00.474178 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:00.473931 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:28:00.474178 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:00.474049 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zzlbv" podUID="73272ce5-1238-42c7-b697-29d21e373fc0" Aug 04 10:28:00.474178 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:00.474137 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rc2x" podUID="b6f16f94-e1b9-4900-a75d-84badd1d9bb7" Aug 04 10:28:00.474178 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:00.473946 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:28:00.474336 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:00.474249 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgpx8" podUID="5436bc5b-0740-4ccb-b4fd-857d4023d8e7" Aug 04 10:28:02.474810 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:02.474357 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:28:02.474810 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:02.474442 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:28:02.474810 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:02.474459 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rc2x" podUID="b6f16f94-e1b9-4900-a75d-84badd1d9bb7" Aug 04 10:28:02.474810 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:02.474578 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zzlbv" podUID="73272ce5-1238-42c7-b697-29d21e373fc0" Aug 04 10:28:02.474810 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:02.474613 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:28:02.474810 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:02.474701 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgpx8" podUID="5436bc5b-0740-4ccb-b4fd-857d4023d8e7" Aug 04 10:28:04.474448 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:04.474166 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:28:04.475038 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:04.474173 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:28:04.475038 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:04.474534 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rc2x" podUID="b6f16f94-e1b9-4900-a75d-84badd1d9bb7" Aug 04 10:28:04.475038 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:04.474236 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:28:04.475038 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:04.474602 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgpx8" podUID="5436bc5b-0740-4ccb-b4fd-857d4023d8e7" Aug 04 10:28:04.475038 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:04.474705 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zzlbv" podUID="73272ce5-1238-42c7-b697-29d21e373fc0" Aug 04 10:28:04.570803 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:04.570775 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6xqqk" event={"ID":"c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9","Type":"ContainerStarted","Data":"e1139c0e036cbddb73dd2686325539d389996b014eebfdec63e12f907d17dae1"} Aug 04 10:28:04.573535 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:04.573511 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" event={"ID":"8c588e91-5d0d-48d2-a8aa-e0231bfbc654","Type":"ContainerStarted","Data":"cdaba5a15c3493f8d4bb6203df81ee40401ebf21fed5326da70f39a28ea10d5d"} Aug 04 10:28:04.573625 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:04.573573 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" event={"ID":"8c588e91-5d0d-48d2-a8aa-e0231bfbc654","Type":"ContainerStarted","Data":"69cb6e241ca68c9c75f1cf105987308ec95fbae2da2544420715fa2a8354ca3b"} Aug 04 10:28:04.573625 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:04.573606 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" event={"ID":"8c588e91-5d0d-48d2-a8aa-e0231bfbc654","Type":"ContainerStarted","Data":"307e034646514cc14c7943f242e8a335b2934ffbe1a63c8e4dca82a031a5076c"} Aug 04 10:28:04.573707 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:04.573647 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" event={"ID":"8c588e91-5d0d-48d2-a8aa-e0231bfbc654","Type":"ContainerStarted","Data":"1ffbbba1f22b722f601ea419cf652536f1fa73aad2d69e9c882bab11fd557d79"} Aug 04 10:28:04.573707 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:04.573692 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" event={"ID":"8c588e91-5d0d-48d2-a8aa-e0231bfbc654","Type":"ContainerStarted","Data":"ac99b9bde21c9d53f573c4210b5e1dcb3bc9e96714f2def6833e6e594b79e30d"} Aug 04 10:28:04.574886 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:04.574864 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s629f" event={"ID":"696380db-a93f-48d7-9bb7-108052391f48","Type":"ContainerStarted","Data":"d42c15f7f2c4c2c44453076556da9536efa718ce254bc44a50e310222c072b54"} Aug 04 10:28:04.576269 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:04.576247 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zpdbz" event={"ID":"9be224ce-6218-4f85-a9f0-a09bf05d4f25","Type":"ContainerStarted","Data":"df0570b67f7bd01769a934364ed002ed2f9c0c83c14947f4c084bfb73313d9a1"} Aug 04 10:28:04.577427 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:04.577409 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5bmt4" event={"ID":"ad9f7eb2-071c-4803-96f1-649155c6f4db","Type":"ContainerStarted","Data":"d2d74f034b73235bf15e9fbccefd5acfa61469dd94568808abcfb3719d7363da"} Aug 04 10:28:04.578723 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:04.578701 2535 generic.go:356] "Generic (PLEG): container finished" podID="7298c7a0-dc2b-4480-aa51-85ba014dcec6" containerID="fe2b4a9716752dd4ff082771e6ce68dca46d6e8e42de707634073e7b0b16b5d8" exitCode=0 Aug 04 10:28:04.578800 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:04.578772 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s59fm" event={"ID":"7298c7a0-dc2b-4480-aa51-85ba014dcec6","Type":"ContainerDied","Data":"fe2b4a9716752dd4ff082771e6ce68dca46d6e8e42de707634073e7b0b16b5d8"} Aug 04 10:28:04.580252 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:04.580231 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" event={"ID":"d6fc23f5-78a7-4129-92ef-b6decff9bf22","Type":"ContainerStarted","Data":"ee73c0f55a669cf964d5ec2308765ec97f0238362514bb73c278eb2a35a84778"} Aug 04 10:28:04.581721 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:04.581704 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zcvc" event={"ID":"8c78f70d-8823-4d6c-96f8-c1318ceddd69","Type":"ContainerStarted","Data":"7ac82b5507f79a3e21c4bda38f1fdd2eb6c5a93f7e1d70804bc9f2b6f0df7c17"} Aug 04 10:28:04.595170 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:04.595132 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal" podStartSLOduration=21.595121867 podStartE2EDuration="21.595121867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 10:27:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:27:47.548686489 +0000 UTC m=+5.522769033" watchObservedRunningTime="2026-08-04 10:28:04.595121867 +0000 UTC m=+22.569204394" Aug 04 10:28:04.595244 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:04.595190 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-6xqqk" podStartSLOduration=3.931985474 podStartE2EDuration="22.595186028s" totalImagesPullingTime="18.663200554s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 10:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:28:04.594765566 +0000 UTC m=+22.568848092" watchObservedRunningTime="2026-08-04 10:28:04.595186028 +0000 UTC m=+22.569268554" Aug 04 10:28:04.640437 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:04.640385 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5bmt4" podStartSLOduration=4.062366048 podStartE2EDuration="22.640368585s" totalImagesPullingTime="18.578002537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:28:04.613028471 +0000 UTC m=+22.587111020" watchObservedRunningTime="2026-08-04 10:28:04.640368585 +0000 UTC m=+22.614451112" Aug 04 10:28:04.663237 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:04.663153 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-s629f" podStartSLOduration=4.089135044 podStartE2EDuration="22.663135609s" totalImagesPullingTime="18.574000565s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 10:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:28:04.663000876 +0000 UTC m=+22.637083407" watchObservedRunningTime="2026-08-04 10:28:04.663135609 +0000 UTC m=+22.637218136" Aug 04 10:28:04.677931 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:04.677910 2535 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5bmt4" Aug 04 10:28:04.678438 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:04.678422 2535 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5bmt4" Aug 04 10:28:04.687144 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:04.684842 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6xm6k" podStartSLOduration=4.076174387 podStartE2EDuration="22.68482465s" totalImagesPullingTime="18.608650263s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 10:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:28:04.682813813 +0000 UTC m=+22.656896342" watchObservedRunningTime="2026-08-04 10:28:04.68482465 +0000 UTC m=+22.658907179" Aug 04 10:28:04.700263 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:04.700208 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zpdbz" podStartSLOduration=12.049202717 podStartE2EDuration="22.700191813s" totalImagesPullingTime="10.650989096s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 10:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:28:04.700021642 +0000 UTC m=+22.674104171" watchObservedRunningTime="2026-08-04 10:28:04.700191813 +0000 UTC m=+22.674274341" Aug 04 10:28:04.733042 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:04.733012 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:04.814110 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:04.814080 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:04.881006 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:04.880972 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:05.505440 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:05.505417 2535 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Aug 04 10:28:05.587230 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:05.587191 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" event={"ID":"8c588e91-5d0d-48d2-a8aa-e0231bfbc654","Type":"ContainerStarted","Data":"997364ef09aa41910001bbb86201b4add4bb34c69557755410049e1f3ff8ad11"} Aug 04 10:28:05.588987 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:05.588955 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-g4z5z" event={"ID":"dfe34fb7-de6d-42c3-b605-5015d4e822c7","Type":"ContainerStarted","Data":"6bebd5785f8ca19985a3dc0dbeb6766cf6b5250d57754889aa6838f52d4af556"} Aug 04 10:28:05.590871 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:05.590832 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zcvc" event={"ID":"8c78f70d-8823-4d6c-96f8-c1318ceddd69","Type":"ContainerStarted","Data":"b7ea0fb7170c76f9368d0d985d72d97551432de29369cfe6ac77ac9c014ee46c"} Aug 04 10:28:05.591456 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:05.591427 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5bmt4" Aug 04 10:28:05.591886 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:05.591867 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5bmt4" Aug 04 10:28:05.606865 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:05.606786 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-g4z5z" podStartSLOduration=5.000972424 podStartE2EDuration="23.606768675s" totalImagesPullingTime="18.605796251s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 10:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:28:05.605994413 +0000 UTC m=+23.580076961" watchObservedRunningTime="2026-08-04 10:28:05.606768675 +0000 UTC m=+23.580851202" Aug 04 10:28:05.628582 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:05.628550 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:05.728921 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:05.728888 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:05.829013 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:05.828978 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:05.969847 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:05.969769 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:06.031504 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:06.031459 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:06.340433 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:06.340398 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:06.460225 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:06.460104 2535 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-04T10:28:05.505435485Z","UUID":"a1e90470-b5dc-44a5-95cd-b74934b95a96","Handler":null,"Name":"","Endpoint":""} Aug 04 10:28:06.461696 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:06.461669 2535 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Aug 04 10:28:06.461796 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:06.461716 2535 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Aug 04 10:28:06.473987 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:06.473963 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:28:06.474090 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:06.473963 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:28:06.474150 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:06.474088 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgpx8" podUID="5436bc5b-0740-4ccb-b4fd-857d4023d8e7" Aug 04 10:28:06.474150 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:06.473963 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:28:06.474242 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:06.474143 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rc2x" podUID="b6f16f94-e1b9-4900-a75d-84badd1d9bb7" Aug 04 10:28:06.474242 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:06.474209 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zzlbv" podUID="73272ce5-1238-42c7-b697-29d21e373fc0" Aug 04 10:28:07.599250 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:07.599000 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" event={"ID":"8c588e91-5d0d-48d2-a8aa-e0231bfbc654","Type":"ContainerStarted","Data":"e76f3fd4e2a2204cc6592a53be2e952b2bc082d2bafec60f3c5180d70f6c1586"} Aug 04 10:28:07.601227 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:07.601193 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zcvc" event={"ID":"8c78f70d-8823-4d6c-96f8-c1318ceddd69","Type":"ContainerStarted","Data":"02c57e60bb055ea5c28c12f081be6f02201762a3bdfbc986d322cb2561a784bd"} Aug 04 10:28:07.619599 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:07.619552 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5zcvc" podStartSLOduration=4.779820963 podStartE2EDuration="25.6195373s" totalImagesPullingTime="20.839716337s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 10:27:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:28:07.619354978 +0000 UTC m=+25.593437508" watchObservedRunningTime="2026-08-04 10:28:07.6195373 +0000 UTC m=+25.593619828" Aug 04 10:28:08.474376 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:08.474341 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:28:08.474595 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:08.474380 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:28:08.474595 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:08.474439 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:28:08.474595 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:08.474469 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rc2x" podUID="b6f16f94-e1b9-4900-a75d-84badd1d9bb7" Aug 04 10:28:08.474595 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:08.474519 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgpx8" podUID="5436bc5b-0740-4ccb-b4fd-857d4023d8e7" Aug 04 10:28:08.474595 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:08.474577 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zzlbv" podUID="73272ce5-1238-42c7-b697-29d21e373fc0" Aug 04 10:28:09.609505 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:09.609298 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" event={"ID":"8c588e91-5d0d-48d2-a8aa-e0231bfbc654","Type":"ContainerStarted","Data":"9b03fa7e6b9bb55219c5c1adfde169e40a335062feb207649349fc174eee8fd2"} Aug 04 10:28:09.610348 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:09.609573 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:28:09.611121 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:09.611099 2535 generic.go:356] "Generic (PLEG): container finished" podID="7298c7a0-dc2b-4480-aa51-85ba014dcec6" containerID="d02ee1cae87edc75412de35685df44d6206a13666e9c613236fe1801b0e6a9c1" exitCode=0 Aug 04 10:28:09.611222 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:09.611136 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s59fm" event={"ID":"7298c7a0-dc2b-4480-aa51-85ba014dcec6","Type":"ContainerDied","Data":"d02ee1cae87edc75412de35685df44d6206a13666e9c613236fe1801b0e6a9c1"} Aug 04 10:28:09.625638 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:09.625619 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:28:09.646090 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:09.646053 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" podStartSLOduration=8.982261211 podStartE2EDuration="27.646038598s" totalImagesPullingTime="18.663777387s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 10:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:28:09.645474534 +0000 UTC m=+27.619557061" watchObservedRunningTime="2026-08-04 10:28:09.646038598 +0000 UTC m=+27.620121124" Aug 04 10:28:10.477586 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:10.477556 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:28:10.477758 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:10.477658 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:28:10.477758 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:10.477667 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:28:10.477758 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:10.477663 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rc2x" podUID="b6f16f94-e1b9-4900-a75d-84badd1d9bb7" Aug 04 10:28:10.477920 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:10.477783 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgpx8" podUID="5436bc5b-0740-4ccb-b4fd-857d4023d8e7" Aug 04 10:28:10.477920 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:10.477876 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zzlbv" podUID="73272ce5-1238-42c7-b697-29d21e373fc0" Aug 04 10:28:10.613725 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:10.613648 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:28:10.614164 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:10.613747 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:28:10.630868 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:10.630841 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:28:11.230344 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:11.230265 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2rc2x"] Aug 04 10:28:11.230501 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:11.230380 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:28:11.230501 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:11.230473 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rc2x" podUID="b6f16f94-e1b9-4900-a75d-84badd1d9bb7" Aug 04 10:28:11.234167 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:11.234143 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qgpx8"] Aug 04 10:28:11.234297 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:11.234245 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:28:11.234374 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:11.234353 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgpx8" podUID="5436bc5b-0740-4ccb-b4fd-857d4023d8e7" Aug 04 10:28:11.244290 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:11.244270 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zzlbv"] Aug 04 10:28:11.244379 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:11.244347 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:28:11.244455 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:11.244435 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zzlbv" podUID="73272ce5-1238-42c7-b697-29d21e373fc0" Aug 04 10:28:11.617526 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:11.617497 2535 generic.go:356] "Generic (PLEG): container finished" podID="7298c7a0-dc2b-4480-aa51-85ba014dcec6" containerID="fb46ffd31db6b3ec16105d25284b60cc136fef07473446daef311ecb070b9a86" exitCode=0 Aug 04 10:28:11.617959 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:11.617583 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s59fm" event={"ID":"7298c7a0-dc2b-4480-aa51-85ba014dcec6","Type":"ContainerDied","Data":"fb46ffd31db6b3ec16105d25284b60cc136fef07473446daef311ecb070b9a86"} Aug 04 10:28:12.474576 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:12.474330 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:28:12.474728 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:12.474671 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rc2x" podUID="b6f16f94-e1b9-4900-a75d-84badd1d9bb7" Aug 04 10:28:13.474101 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:13.474070 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:28:13.474459 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:13.474175 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zzlbv" podUID="73272ce5-1238-42c7-b697-29d21e373fc0" Aug 04 10:28:13.474459 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:13.474224 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:28:13.474459 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:13.474318 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgpx8" podUID="5436bc5b-0740-4ccb-b4fd-857d4023d8e7" Aug 04 10:28:13.623755 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:13.623725 2535 generic.go:356] "Generic (PLEG): container finished" podID="7298c7a0-dc2b-4480-aa51-85ba014dcec6" containerID="fb60b077d34a67edc631574a58086d694bc68a43c151a00e4afe1fdd08a3da09" exitCode=0 Aug 04 10:28:13.623887 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:13.623760 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s59fm" event={"ID":"7298c7a0-dc2b-4480-aa51-85ba014dcec6","Type":"ContainerDied","Data":"fb60b077d34a67edc631574a58086d694bc68a43c151a00e4afe1fdd08a3da09"} Aug 04 10:28:14.477820 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:14.477782 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:28:14.478363 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:14.477932 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rc2x" podUID="b6f16f94-e1b9-4900-a75d-84badd1d9bb7" Aug 04 10:28:15.407222 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:15.407184 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:15.474465 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:15.474434 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:28:15.474651 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:15.474432 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:28:15.474651 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:15.474586 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qgpx8" podUID="5436bc5b-0740-4ccb-b4fd-857d4023d8e7" Aug 04 10:28:15.474651 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:15.474611 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zzlbv" podUID="73272ce5-1238-42c7-b697-29d21e373fc0" Aug 04 10:28:15.499744 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:15.499709 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:15.856647 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:15.856615 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:15.914907 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:15.914871 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:16.145079 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:16.144995 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5436bc5b-0740-4ccb-b4fd-857d4023d8e7-metrics-certs\") pod \"network-metrics-daemon-qgpx8\" (UID: \"5436bc5b-0740-4ccb-b4fd-857d4023d8e7\") " pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:28:16.145079 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:16.145057 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73272ce5-1238-42c7-b697-29d21e373fc0-original-pull-secret\") pod \"global-pull-secret-syncer-zzlbv\" (UID: \"73272ce5-1238-42c7-b697-29d21e373fc0\") " pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:28:16.145380 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:16.145162 2535 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 10:28:16.145380 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:16.145237 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5436bc5b-0740-4ccb-b4fd-857d4023d8e7-metrics-certs podName:5436bc5b-0740-4ccb-b4fd-857d4023d8e7 nodeName:}" failed. No retries permitted until 2026-08-04 10:28:48.145214481 +0000 UTC m=+66.119296991 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5436bc5b-0740-4ccb-b4fd-857d4023d8e7-metrics-certs") pod "network-metrics-daemon-qgpx8" (UID: "5436bc5b-0740-4ccb-b4fd-857d4023d8e7") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 10:28:16.145380 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:16.145162 2535 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 10:28:16.145380 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:16.145337 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73272ce5-1238-42c7-b697-29d21e373fc0-original-pull-secret podName:73272ce5-1238-42c7-b697-29d21e373fc0 nodeName:}" failed. No retries permitted until 2026-08-04 10:28:48.145315638 +0000 UTC m=+66.119398160 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/73272ce5-1238-42c7-b697-29d21e373fc0-original-pull-secret") pod "global-pull-secret-syncer-zzlbv" (UID: "73272ce5-1238-42c7-b697-29d21e373fc0") : object "kube-system"/"original-pull-secret" not registered Aug 04 10:28:16.346586 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:16.346547 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w28sh\" (UniqueName: \"kubernetes.io/projected/b6f16f94-e1b9-4900-a75d-84badd1d9bb7-kube-api-access-w28sh\") pod \"network-check-target-2rc2x\" (UID: \"b6f16f94-e1b9-4900-a75d-84badd1d9bb7\") " pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:28:16.346769 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:16.346730 2535 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 10:28:16.346769 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:16.346755 2535 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 10:28:16.346769 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:16.346768 2535 projected.go:196] Error preparing data for projected volume kube-api-access-w28sh for pod openshift-network-diagnostics/network-check-target-2rc2x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 10:28:16.346972 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:16.346820 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b6f16f94-e1b9-4900-a75d-84badd1d9bb7-kube-api-access-w28sh podName:b6f16f94-e1b9-4900-a75d-84badd1d9bb7 nodeName:}" failed. No retries permitted until 2026-08-04 10:28:48.346803217 +0000 UTC m=+66.320885735 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-w28sh" (UniqueName: "kubernetes.io/projected/b6f16f94-e1b9-4900-a75d-84badd1d9bb7-kube-api-access-w28sh") pod "network-check-target-2rc2x" (UID: "b6f16f94-e1b9-4900-a75d-84badd1d9bb7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 10:28:16.477852 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:16.477826 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:28:16.477978 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:16.477956 2535 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rc2x" podUID="b6f16f94-e1b9-4900-a75d-84badd1d9bb7" Aug 04 10:28:16.840650 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:16.840617 2535 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeReady" Aug 04 10:28:16.841202 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:16.840831 2535 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Aug 04 10:28:16.886603 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:16.886571 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7wdtv"] Aug 04 10:28:16.891784 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:16.891760 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ttzdq"] Aug 04 10:28:16.891927 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:16.891887 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7wdtv" Aug 04 10:28:16.893795 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:16.893725 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Aug 04 10:28:16.893795 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:16.893786 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Aug 04 10:28:16.893994 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:16.893833 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wlm2z\"" Aug 04 10:28:16.895501 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:16.895440 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ttzdq" Aug 04 10:28:16.897239 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:16.897220 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8cx65\"" Aug 04 10:28:16.897444 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:16.897400 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Aug 04 10:28:16.897558 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:16.897447 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Aug 04 10:28:16.897558 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:16.897514 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Aug 04 10:28:16.900059 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:16.899886 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ttzdq"] Aug 04 10:28:16.906261 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:16.906239 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7wdtv"] Aug 04 10:28:17.025132 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:17.025100 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:17.051014 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:17.050988 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzqpw\" (UniqueName: \"kubernetes.io/projected/1955136f-4af9-4ab3-81df-790fa97cddab-kube-api-access-wzqpw\") pod \"ingress-canary-ttzdq\" (UID: \"1955136f-4af9-4ab3-81df-790fa97cddab\") " pod="openshift-ingress-canary/ingress-canary-ttzdq" Aug 04 10:28:17.051167 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:17.051059 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2hsp\" (UniqueName: \"kubernetes.io/projected/5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99-kube-api-access-m2hsp\") pod \"dns-default-7wdtv\" (UID: \"5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99\") " pod="openshift-dns/dns-default-7wdtv" Aug 04 10:28:17.051167 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:17.051143 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99-config-volume\") pod \"dns-default-7wdtv\" (UID: \"5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99\") " pod="openshift-dns/dns-default-7wdtv" Aug 04 10:28:17.051269 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:17.051173 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1955136f-4af9-4ab3-81df-790fa97cddab-cert\") pod \"ingress-canary-ttzdq\" (UID: \"1955136f-4af9-4ab3-81df-790fa97cddab\") " pod="openshift-ingress-canary/ingress-canary-ttzdq" Aug 04 10:28:17.051269 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:17.051205 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99-metrics-tls\") pod \"dns-default-7wdtv\" (UID: \"5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99\") " pod="openshift-dns/dns-default-7wdtv" Aug 04 10:28:17.051362 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:17.051266 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99-tmp-dir\") pod \"dns-default-7wdtv\" (UID: \"5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99\") " pod="openshift-dns/dns-default-7wdtv" Aug 04 10:28:17.126911 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:17.126829 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:17.151937 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:17.151907 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2hsp\" (UniqueName: \"kubernetes.io/projected/5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99-kube-api-access-m2hsp\") pod \"dns-default-7wdtv\" (UID: \"5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99\") " pod="openshift-dns/dns-default-7wdtv" Aug 04 10:28:17.151937 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:17.151942 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99-config-volume\") pod \"dns-default-7wdtv\" (UID: \"5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99\") " pod="openshift-dns/dns-default-7wdtv" Aug 04 10:28:17.152143 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:17.151964 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1955136f-4af9-4ab3-81df-790fa97cddab-cert\") pod \"ingress-canary-ttzdq\" (UID: \"1955136f-4af9-4ab3-81df-790fa97cddab\") " pod="openshift-ingress-canary/ingress-canary-ttzdq" Aug 04 10:28:17.152143 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:17.151990 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99-metrics-tls\") pod \"dns-default-7wdtv\" (UID: \"5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99\") " pod="openshift-dns/dns-default-7wdtv" Aug 04 10:28:17.152143 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:17.152018 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99-tmp-dir\") pod \"dns-default-7wdtv\" (UID: \"5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99\") " pod="openshift-dns/dns-default-7wdtv" Aug 04 10:28:17.152294 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:17.152265 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzqpw\" (UniqueName: \"kubernetes.io/projected/1955136f-4af9-4ab3-81df-790fa97cddab-kube-api-access-wzqpw\") pod \"ingress-canary-ttzdq\" (UID: \"1955136f-4af9-4ab3-81df-790fa97cddab\") " pod="openshift-ingress-canary/ingress-canary-ttzdq" Aug 04 10:28:17.152338 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:17.152303 2535 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 10:28:17.152380 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:17.152364 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99-metrics-tls podName:5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99 nodeName:}" failed. No retries permitted until 2026-08-04 10:28:17.652343705 +0000 UTC m=+35.626426227 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99-metrics-tls") pod "dns-default-7wdtv" (UID: "5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99") : secret "dns-default-metrics-tls" not found Aug 04 10:28:17.152434 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:17.152389 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99-tmp-dir\") pod \"dns-default-7wdtv\" (UID: \"5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99\") " pod="openshift-dns/dns-default-7wdtv" Aug 04 10:28:17.152785 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:17.152755 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99-config-volume\") pod \"dns-default-7wdtv\" (UID: \"5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99\") " pod="openshift-dns/dns-default-7wdtv" Aug 04 10:28:17.157014 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:17.156992 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1955136f-4af9-4ab3-81df-790fa97cddab-cert\") pod \"ingress-canary-ttzdq\" (UID: \"1955136f-4af9-4ab3-81df-790fa97cddab\") " pod="openshift-ingress-canary/ingress-canary-ttzdq" Aug 04 10:28:17.162737 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:17.162706 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2hsp\" (UniqueName: \"kubernetes.io/projected/5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99-kube-api-access-m2hsp\") pod \"dns-default-7wdtv\" (UID: \"5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99\") " pod="openshift-dns/dns-default-7wdtv" Aug 04 10:28:17.162848 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:17.162830 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzqpw\" (UniqueName: \"kubernetes.io/projected/1955136f-4af9-4ab3-81df-790fa97cddab-kube-api-access-wzqpw\") pod \"ingress-canary-ttzdq\" (UID: \"1955136f-4af9-4ab3-81df-790fa97cddab\") " pod="openshift-ingress-canary/ingress-canary-ttzdq" Aug 04 10:28:17.213809 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:17.213782 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ttzdq" Aug 04 10:28:17.473964 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:17.473883 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:28:17.474123 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:17.473883 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:28:17.476439 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:17.476411 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Aug 04 10:28:17.477071 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:17.476937 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Aug 04 10:28:17.477071 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:17.476942 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-p48mn\"" Aug 04 10:28:17.641342 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:17.640775 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ttzdq"] Aug 04 10:28:17.646740 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:28:17.646717 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1955136f_4af9_4ab3_81df_790fa97cddab.slice/crio-de2a1491c2d329cace4eef36417dd180f5a08e75cfd9381089e3e752802b6544 WatchSource:0}: Error finding container de2a1491c2d329cace4eef36417dd180f5a08e75cfd9381089e3e752802b6544: Status 404 returned error can't find the container with id de2a1491c2d329cace4eef36417dd180f5a08e75cfd9381089e3e752802b6544 Aug 04 10:28:17.656425 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:17.656397 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99-metrics-tls\") pod \"dns-default-7wdtv\" (UID: \"5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99\") " pod="openshift-dns/dns-default-7wdtv" Aug 04 10:28:17.656584 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:17.656566 2535 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 10:28:17.656638 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:17.656629 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99-metrics-tls podName:5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99 nodeName:}" failed. No retries permitted until 2026-08-04 10:28:18.656609794 +0000 UTC m=+36.630692309 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99-metrics-tls") pod "dns-default-7wdtv" (UID: "5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99") : secret "dns-default-metrics-tls" not found Aug 04 10:28:17.935629 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:17.935597 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:17.995147 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:17.995079 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:18.447154 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:18.447120 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:18.474501 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:18.474467 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:28:18.476898 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:18.476876 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Aug 04 10:28:18.477349 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:18.477332 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Aug 04 10:28:18.477447 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:18.477366 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-clvkl\"" Aug 04 10:28:18.547851 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:18.547824 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:18.639350 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:18.639319 2535 generic.go:356] "Generic (PLEG): container finished" podID="7298c7a0-dc2b-4480-aa51-85ba014dcec6" containerID="6cc0784e579ef95a2ca44557263f2812dae8ca2a9f458ea2cfe49fe2b8b022a5" exitCode=0 Aug 04 10:28:18.639530 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:18.639420 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s59fm" event={"ID":"7298c7a0-dc2b-4480-aa51-85ba014dcec6","Type":"ContainerDied","Data":"6cc0784e579ef95a2ca44557263f2812dae8ca2a9f458ea2cfe49fe2b8b022a5"} Aug 04 10:28:18.640905 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:18.640643 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ttzdq" event={"ID":"1955136f-4af9-4ab3-81df-790fa97cddab","Type":"ContainerStarted","Data":"de2a1491c2d329cace4eef36417dd180f5a08e75cfd9381089e3e752802b6544"} Aug 04 10:28:18.664242 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:18.664214 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99-metrics-tls\") pod \"dns-default-7wdtv\" (UID: \"5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99\") " pod="openshift-dns/dns-default-7wdtv" Aug 04 10:28:18.664423 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:18.664408 2535 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 10:28:18.664500 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:18.664474 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99-metrics-tls podName:5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99 nodeName:}" failed. No retries permitted until 2026-08-04 10:28:20.664456146 +0000 UTC m=+38.638538669 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99-metrics-tls") pod "dns-default-7wdtv" (UID: "5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99") : secret "dns-default-metrics-tls" not found Aug 04 10:28:19.646992 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:19.646954 2535 generic.go:356] "Generic (PLEG): container finished" podID="7298c7a0-dc2b-4480-aa51-85ba014dcec6" containerID="7d6ab9f3df91f0cba7b400330ed377a461a2aee608d119aba631bbb3752a178d" exitCode=0 Aug 04 10:28:19.647436 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:19.647004 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s59fm" event={"ID":"7298c7a0-dc2b-4480-aa51-85ba014dcec6","Type":"ContainerDied","Data":"7d6ab9f3df91f0cba7b400330ed377a461a2aee608d119aba631bbb3752a178d"} Aug 04 10:28:20.562014 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:20.561821 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:20.621957 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:20.621930 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:20.652166 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:20.652137 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s59fm" event={"ID":"7298c7a0-dc2b-4480-aa51-85ba014dcec6","Type":"ContainerStarted","Data":"b890b8ed020d660908cea6da7c8eb76bae208205c835b8483716761d543bd310"} Aug 04 10:28:20.653520 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:20.653481 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ttzdq" event={"ID":"1955136f-4af9-4ab3-81df-790fa97cddab","Type":"ContainerStarted","Data":"bb838f034ecd3eddf2fa25a006fdca7a1e5b61d2f0104abaaf9825d2c35e8d81"} Aug 04 10:28:20.678746 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:20.678721 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99-metrics-tls\") pod \"dns-default-7wdtv\" (UID: \"5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99\") " pod="openshift-dns/dns-default-7wdtv" Aug 04 10:28:20.678883 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:20.678866 2535 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 10:28:20.678937 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:20.678928 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99-metrics-tls podName:5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99 nodeName:}" failed. No retries permitted until 2026-08-04 10:28:24.678912381 +0000 UTC m=+42.652994885 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99-metrics-tls") pod "dns-default-7wdtv" (UID: "5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99") : secret "dns-default-metrics-tls" not found Aug 04 10:28:20.682856 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:20.682822 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-s59fm" podStartSLOduration=9.534593803 podStartE2EDuration="38.682808686s" totalImagesPullingTime="29.148214883s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 10:27:42 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:28:20.681576531 +0000 UTC m=+38.655659079" watchObservedRunningTime="2026-08-04 10:28:20.682808686 +0000 UTC m=+38.656891212" Aug 04 10:28:20.698909 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:20.698849 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ttzdq" podStartSLOduration=2.314074651 podStartE2EDuration="4.698833878s" totalImagesPullingTime="2.384759227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:28:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:28:20.698634073 +0000 UTC m=+38.672716622" watchObservedRunningTime="2026-08-04 10:28:20.698833878 +0000 UTC m=+38.672916405" Aug 04 10:28:20.803142 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:20.803115 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:20.882112 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:20.882038 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:20.976301 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:20.976269 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:21.037557 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:21.037526 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:21.138569 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:21.138477 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:21.198697 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:21.198672 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:22.057602 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.057566 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:22.069208 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.069185 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tgpbn"] Aug 04 10:28:22.086948 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.086927 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tgpbn"] Aug 04 10:28:22.087057 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.087040 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tgpbn" Aug 04 10:28:22.088859 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.088836 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Aug 04 10:28:22.088954 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.088890 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Aug 04 10:28:22.089171 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.089156 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ntlkp\"" Aug 04 10:28:22.089271 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.089257 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Aug 04 10:28:22.089340 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.089294 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Aug 04 10:28:22.125537 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.125510 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:22.189640 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.189616 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/51786990-2492-4142-994e-9e908bd478c1-data-volume\") pod \"insights-runtime-extractor-tgpbn\" (UID: \"51786990-2492-4142-994e-9e908bd478c1\") " pod="openshift-insights/insights-runtime-extractor-tgpbn" Aug 04 10:28:22.189740 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.189675 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/51786990-2492-4142-994e-9e908bd478c1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tgpbn\" (UID: \"51786990-2492-4142-994e-9e908bd478c1\") " pod="openshift-insights/insights-runtime-extractor-tgpbn" Aug 04 10:28:22.189784 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.189740 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/51786990-2492-4142-994e-9e908bd478c1-crio-socket\") pod \"insights-runtime-extractor-tgpbn\" (UID: \"51786990-2492-4142-994e-9e908bd478c1\") " pod="openshift-insights/insights-runtime-extractor-tgpbn" Aug 04 10:28:22.189784 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.189776 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/51786990-2492-4142-994e-9e908bd478c1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tgpbn\" (UID: \"51786990-2492-4142-994e-9e908bd478c1\") " pod="openshift-insights/insights-runtime-extractor-tgpbn" Aug 04 10:28:22.189849 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.189830 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4lj6\" (UniqueName: \"kubernetes.io/projected/51786990-2492-4142-994e-9e908bd478c1-kube-api-access-w4lj6\") pod \"insights-runtime-extractor-tgpbn\" (UID: \"51786990-2492-4142-994e-9e908bd478c1\") " pod="openshift-insights/insights-runtime-extractor-tgpbn" Aug 04 10:28:22.193891 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.193869 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:22.249372 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.249345 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-6cskx"] Aug 04 10:28:22.274251 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.274231 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:22.275219 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.275203 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-6cskx"] Aug 04 10:28:22.275294 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.275284 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-6cskx" Aug 04 10:28:22.277677 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.277655 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Aug 04 10:28:22.277896 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.277756 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-d7fbm\"" Aug 04 10:28:22.277896 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.277814 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Aug 04 10:28:22.290740 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.290720 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/51786990-2492-4142-994e-9e908bd478c1-data-volume\") pod \"insights-runtime-extractor-tgpbn\" (UID: \"51786990-2492-4142-994e-9e908bd478c1\") " pod="openshift-insights/insights-runtime-extractor-tgpbn" Aug 04 10:28:22.290843 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.290774 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/51786990-2492-4142-994e-9e908bd478c1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tgpbn\" (UID: \"51786990-2492-4142-994e-9e908bd478c1\") " pod="openshift-insights/insights-runtime-extractor-tgpbn" Aug 04 10:28:22.290843 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.290808 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/51786990-2492-4142-994e-9e908bd478c1-crio-socket\") pod \"insights-runtime-extractor-tgpbn\" (UID: \"51786990-2492-4142-994e-9e908bd478c1\") " pod="openshift-insights/insights-runtime-extractor-tgpbn" Aug 04 10:28:22.290843 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.290825 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/51786990-2492-4142-994e-9e908bd478c1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tgpbn\" (UID: \"51786990-2492-4142-994e-9e908bd478c1\") " pod="openshift-insights/insights-runtime-extractor-tgpbn" Aug 04 10:28:22.291007 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.290855 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4lj6\" (UniqueName: \"kubernetes.io/projected/51786990-2492-4142-994e-9e908bd478c1-kube-api-access-w4lj6\") pod \"insights-runtime-extractor-tgpbn\" (UID: \"51786990-2492-4142-994e-9e908bd478c1\") " pod="openshift-insights/insights-runtime-extractor-tgpbn" Aug 04 10:28:22.291007 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:22.290921 2535 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 10:28:22.291007 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:22.290950 2535 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 10:28:22.291007 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:22.290979 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51786990-2492-4142-994e-9e908bd478c1-insights-runtime-extractor-tls podName:51786990-2492-4142-994e-9e908bd478c1 nodeName:}" failed. No retries permitted until 2026-08-04 10:28:22.790960795 +0000 UTC m=+40.765043300 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/51786990-2492-4142-994e-9e908bd478c1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tgpbn" (UID: "51786990-2492-4142-994e-9e908bd478c1") : secret "insights-runtime-extractor-tls" not found Aug 04 10:28:22.291181 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:22.291021 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/51786990-2492-4142-994e-9e908bd478c1-kube-rbac-proxy-cm podName:51786990-2492-4142-994e-9e908bd478c1 nodeName:}" failed. No retries permitted until 2026-08-04 10:28:22.791004085 +0000 UTC m=+40.765086613 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/51786990-2492-4142-994e-9e908bd478c1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tgpbn" (UID: "51786990-2492-4142-994e-9e908bd478c1") : configmap "kube-rbac-proxy" not found Aug 04 10:28:22.291181 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.291064 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/51786990-2492-4142-994e-9e908bd478c1-crio-socket\") pod \"insights-runtime-extractor-tgpbn\" (UID: \"51786990-2492-4142-994e-9e908bd478c1\") " pod="openshift-insights/insights-runtime-extractor-tgpbn" Aug 04 10:28:22.291181 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.291085 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/51786990-2492-4142-994e-9e908bd478c1-data-volume\") pod \"insights-runtime-extractor-tgpbn\" (UID: \"51786990-2492-4142-994e-9e908bd478c1\") " pod="openshift-insights/insights-runtime-extractor-tgpbn" Aug 04 10:28:22.300183 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.300161 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4lj6\" (UniqueName: \"kubernetes.io/projected/51786990-2492-4142-994e-9e908bd478c1-kube-api-access-w4lj6\") pod \"insights-runtime-extractor-tgpbn\" (UID: \"51786990-2492-4142-994e-9e908bd478c1\") " pod="openshift-insights/insights-runtime-extractor-tgpbn" Aug 04 10:28:22.339269 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.339218 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:22.392084 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.392063 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vw8b\" (UniqueName: \"kubernetes.io/projected/da80e736-cae4-4cde-a20e-e54d05afe4b7-kube-api-access-7vw8b\") pod \"migrator-68bc745d87-6cskx\" (UID: \"da80e736-cae4-4cde-a20e-e54d05afe4b7\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-6cskx" Aug 04 10:28:22.492433 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.492407 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vw8b\" (UniqueName: \"kubernetes.io/projected/da80e736-cae4-4cde-a20e-e54d05afe4b7-kube-api-access-7vw8b\") pod \"migrator-68bc745d87-6cskx\" (UID: \"da80e736-cae4-4cde-a20e-e54d05afe4b7\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-6cskx" Aug 04 10:28:22.503674 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.503653 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vw8b\" (UniqueName: \"kubernetes.io/projected/da80e736-cae4-4cde-a20e-e54d05afe4b7-kube-api-access-7vw8b\") pod \"migrator-68bc745d87-6cskx\" (UID: \"da80e736-cae4-4cde-a20e-e54d05afe4b7\") " pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-6cskx" Aug 04 10:28:22.559301 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.559272 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:22.586764 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.586739 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-6cskx" Aug 04 10:28:22.619070 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.619042 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:22.736631 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.736600 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-68bc745d87-6cskx"] Aug 04 10:28:22.739653 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:28:22.739622 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda80e736_cae4_4cde_a20e_e54d05afe4b7.slice/crio-1acb86c225b37d828cdb8cbf9e9c522a277f1a16674ee67379b93e2f79e856c6 WatchSource:0}: Error finding container 1acb86c225b37d828cdb8cbf9e9c522a277f1a16674ee67379b93e2f79e856c6: Status 404 returned error can't find the container with id 1acb86c225b37d828cdb8cbf9e9c522a277f1a16674ee67379b93e2f79e856c6 Aug 04 10:28:22.794569 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.794547 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/51786990-2492-4142-994e-9e908bd478c1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tgpbn\" (UID: \"51786990-2492-4142-994e-9e908bd478c1\") " pod="openshift-insights/insights-runtime-extractor-tgpbn" Aug 04 10:28:22.794669 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.794626 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/51786990-2492-4142-994e-9e908bd478c1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tgpbn\" (UID: \"51786990-2492-4142-994e-9e908bd478c1\") " pod="openshift-insights/insights-runtime-extractor-tgpbn" Aug 04 10:28:22.794708 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:22.794663 2535 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 10:28:22.794741 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:22.794719 2535 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 10:28:22.794741 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:22.794735 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/51786990-2492-4142-994e-9e908bd478c1-kube-rbac-proxy-cm podName:51786990-2492-4142-994e-9e908bd478c1 nodeName:}" failed. No retries permitted until 2026-08-04 10:28:23.794718089 +0000 UTC m=+41.768800595 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/51786990-2492-4142-994e-9e908bd478c1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tgpbn" (UID: "51786990-2492-4142-994e-9e908bd478c1") : configmap "kube-rbac-proxy" not found Aug 04 10:28:22.794819 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:22.794754 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51786990-2492-4142-994e-9e908bd478c1-insights-runtime-extractor-tls podName:51786990-2492-4142-994e-9e908bd478c1 nodeName:}" failed. No retries permitted until 2026-08-04 10:28:23.794743612 +0000 UTC m=+41.768826118 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/51786990-2492-4142-994e-9e908bd478c1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tgpbn" (UID: "51786990-2492-4142-994e-9e908bd478c1") : secret "insights-runtime-extractor-tls" not found Aug 04 10:28:22.854069 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.854040 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:22.928857 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:22.928801 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:23.662422 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:23.662388 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-6cskx" event={"ID":"da80e736-cae4-4cde-a20e-e54d05afe4b7","Type":"ContainerStarted","Data":"1acb86c225b37d828cdb8cbf9e9c522a277f1a16674ee67379b93e2f79e856c6"} Aug 04 10:28:23.802857 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:23.802819 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/51786990-2492-4142-994e-9e908bd478c1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tgpbn\" (UID: \"51786990-2492-4142-994e-9e908bd478c1\") " pod="openshift-insights/insights-runtime-extractor-tgpbn" Aug 04 10:28:23.803036 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:23.802885 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/51786990-2492-4142-994e-9e908bd478c1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tgpbn\" (UID: \"51786990-2492-4142-994e-9e908bd478c1\") " pod="openshift-insights/insights-runtime-extractor-tgpbn" Aug 04 10:28:23.803036 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:23.802969 2535 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 10:28:23.803036 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:23.802999 2535 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 10:28:23.803036 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:23.803034 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51786990-2492-4142-994e-9e908bd478c1-insights-runtime-extractor-tls podName:51786990-2492-4142-994e-9e908bd478c1 nodeName:}" failed. No retries permitted until 2026-08-04 10:28:25.803016714 +0000 UTC m=+43.777099219 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/51786990-2492-4142-994e-9e908bd478c1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tgpbn" (UID: "51786990-2492-4142-994e-9e908bd478c1") : secret "insights-runtime-extractor-tls" not found Aug 04 10:28:23.803227 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:23.803052 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/51786990-2492-4142-994e-9e908bd478c1-kube-rbac-proxy-cm podName:51786990-2492-4142-994e-9e908bd478c1 nodeName:}" failed. No retries permitted until 2026-08-04 10:28:25.803039837 +0000 UTC m=+43.777122347 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/51786990-2492-4142-994e-9e908bd478c1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tgpbn" (UID: "51786990-2492-4142-994e-9e908bd478c1") : configmap "kube-rbac-proxy" not found Aug 04 10:28:24.203789 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:24.203765 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s629f_696380db-a93f-48d7-9bb7-108052391f48/dns-node-resolver/0.log" Aug 04 10:28:24.456309 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:24.456265 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-787tx"] Aug 04 10:28:24.486901 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:24.486881 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-787tx"] Aug 04 10:28:24.487010 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:24.486994 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-787tx" Aug 04 10:28:24.489472 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:24.489456 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Aug 04 10:28:24.489589 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:24.489476 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-jkpb9\"" Aug 04 10:28:24.489850 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:24.489834 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Aug 04 10:28:24.489938 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:24.489898 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Aug 04 10:28:24.489992 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:24.489951 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Aug 04 10:28:24.498660 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:24.498641 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:24.563296 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:24.563268 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:24.608277 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:24.608246 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ttm8\" (UniqueName: \"kubernetes.io/projected/71172b55-f0af-407d-bb6f-89f2d92af16e-kube-api-access-2ttm8\") pod \"service-ca-98c88fbb5-787tx\" (UID: \"71172b55-f0af-407d-bb6f-89f2d92af16e\") " pod="openshift-service-ca/service-ca-98c88fbb5-787tx" Aug 04 10:28:24.608396 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:24.608283 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/71172b55-f0af-407d-bb6f-89f2d92af16e-signing-key\") pod \"service-ca-98c88fbb5-787tx\" (UID: \"71172b55-f0af-407d-bb6f-89f2d92af16e\") " pod="openshift-service-ca/service-ca-98c88fbb5-787tx" Aug 04 10:28:24.608396 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:24.608311 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/71172b55-f0af-407d-bb6f-89f2d92af16e-signing-cabundle\") pod \"service-ca-98c88fbb5-787tx\" (UID: \"71172b55-f0af-407d-bb6f-89f2d92af16e\") " pod="openshift-service-ca/service-ca-98c88fbb5-787tx" Aug 04 10:28:24.666762 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:24.666735 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-6cskx" event={"ID":"da80e736-cae4-4cde-a20e-e54d05afe4b7","Type":"ContainerStarted","Data":"731e3d01d2577830d2797f36073602e17231e56563358317cb1fdc6a652ad4ea"} Aug 04 10:28:24.667047 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:24.666781 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-6cskx" event={"ID":"da80e736-cae4-4cde-a20e-e54d05afe4b7","Type":"ContainerStarted","Data":"deec9554687e6ece4d8ec913723cb39a4870c8ac73587a55e05ed4106774668b"} Aug 04 10:28:24.694674 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:24.694600 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-68bc745d87-6cskx" podStartSLOduration=1.1202842020000001 podStartE2EDuration="2.694580221s" totalImagesPullingTime="1.574296019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:28:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:28:24.69367729 +0000 UTC m=+42.667759817" watchObservedRunningTime="2026-08-04 10:28:24.694580221 +0000 UTC m=+42.668662749" Aug 04 10:28:24.709509 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:24.709427 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99-metrics-tls\") pod \"dns-default-7wdtv\" (UID: \"5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99\") " pod="openshift-dns/dns-default-7wdtv" Aug 04 10:28:24.709509 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:24.709460 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ttm8\" (UniqueName: \"kubernetes.io/projected/71172b55-f0af-407d-bb6f-89f2d92af16e-kube-api-access-2ttm8\") pod \"service-ca-98c88fbb5-787tx\" (UID: \"71172b55-f0af-407d-bb6f-89f2d92af16e\") " pod="openshift-service-ca/service-ca-98c88fbb5-787tx" Aug 04 10:28:24.709663 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:24.709525 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/71172b55-f0af-407d-bb6f-89f2d92af16e-signing-key\") pod \"service-ca-98c88fbb5-787tx\" (UID: \"71172b55-f0af-407d-bb6f-89f2d92af16e\") " pod="openshift-service-ca/service-ca-98c88fbb5-787tx" Aug 04 10:28:24.709663 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:24.709568 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/71172b55-f0af-407d-bb6f-89f2d92af16e-signing-cabundle\") pod \"service-ca-98c88fbb5-787tx\" (UID: \"71172b55-f0af-407d-bb6f-89f2d92af16e\") " pod="openshift-service-ca/service-ca-98c88fbb5-787tx" Aug 04 10:28:24.709663 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:24.709641 2535 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 10:28:24.709770 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:24.709719 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99-metrics-tls podName:5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99 nodeName:}" failed. No retries permitted until 2026-08-04 10:28:32.709699404 +0000 UTC m=+50.683781924 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99-metrics-tls") pod "dns-default-7wdtv" (UID: "5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99") : secret "dns-default-metrics-tls" not found Aug 04 10:28:24.710297 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:24.710281 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/71172b55-f0af-407d-bb6f-89f2d92af16e-signing-cabundle\") pod \"service-ca-98c88fbb5-787tx\" (UID: \"71172b55-f0af-407d-bb6f-89f2d92af16e\") " pod="openshift-service-ca/service-ca-98c88fbb5-787tx" Aug 04 10:28:24.711775 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:24.711758 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/71172b55-f0af-407d-bb6f-89f2d92af16e-signing-key\") pod \"service-ca-98c88fbb5-787tx\" (UID: \"71172b55-f0af-407d-bb6f-89f2d92af16e\") " pod="openshift-service-ca/service-ca-98c88fbb5-787tx" Aug 04 10:28:24.720961 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:24.720920 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ttm8\" (UniqueName: \"kubernetes.io/projected/71172b55-f0af-407d-bb6f-89f2d92af16e-kube-api-access-2ttm8\") pod \"service-ca-98c88fbb5-787tx\" (UID: \"71172b55-f0af-407d-bb6f-89f2d92af16e\") " pod="openshift-service-ca/service-ca-98c88fbb5-787tx" Aug 04 10:28:24.783829 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:24.783808 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:24.795802 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:24.795784 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-787tx" Aug 04 10:28:24.850045 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:24.850016 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:24.934412 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:24.934380 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-787tx"] Aug 04 10:28:24.937785 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:28:24.937758 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71172b55_f0af_407d_bb6f_89f2d92af16e.slice/crio-96265d1f876dfccff5a43731f3dc1829e1ea14a5ba8556662f1ae8cb155c56fe WatchSource:0}: Error finding container 96265d1f876dfccff5a43731f3dc1829e1ea14a5ba8556662f1ae8cb155c56fe: Status 404 returned error can't find the container with id 96265d1f876dfccff5a43731f3dc1829e1ea14a5ba8556662f1ae8cb155c56fe Aug 04 10:28:25.209901 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:25.209874 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zpdbz_9be224ce-6218-4f85-a9f0-a09bf05d4f25/node-ca/0.log" Aug 04 10:28:25.393359 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:25.393328 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:25.671232 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:25.671196 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-787tx" event={"ID":"71172b55-f0af-407d-bb6f-89f2d92af16e","Type":"ContainerStarted","Data":"96265d1f876dfccff5a43731f3dc1829e1ea14a5ba8556662f1ae8cb155c56fe"} Aug 04 10:28:25.817425 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:25.817388 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/51786990-2492-4142-994e-9e908bd478c1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tgpbn\" (UID: \"51786990-2492-4142-994e-9e908bd478c1\") " pod="openshift-insights/insights-runtime-extractor-tgpbn" Aug 04 10:28:25.817601 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:25.817530 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/51786990-2492-4142-994e-9e908bd478c1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tgpbn\" (UID: \"51786990-2492-4142-994e-9e908bd478c1\") " pod="openshift-insights/insights-runtime-extractor-tgpbn" Aug 04 10:28:25.817601 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:25.817551 2535 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 10:28:25.817721 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:25.817627 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/51786990-2492-4142-994e-9e908bd478c1-kube-rbac-proxy-cm podName:51786990-2492-4142-994e-9e908bd478c1 nodeName:}" failed. No retries permitted until 2026-08-04 10:28:29.817606413 +0000 UTC m=+47.791688923 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/51786990-2492-4142-994e-9e908bd478c1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tgpbn" (UID: "51786990-2492-4142-994e-9e908bd478c1") : configmap "kube-rbac-proxy" not found Aug 04 10:28:25.817721 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:25.817638 2535 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 10:28:25.817721 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:25.817689 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51786990-2492-4142-994e-9e908bd478c1-insights-runtime-extractor-tls podName:51786990-2492-4142-994e-9e908bd478c1 nodeName:}" failed. No retries permitted until 2026-08-04 10:28:29.817672685 +0000 UTC m=+47.791755192 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/51786990-2492-4142-994e-9e908bd478c1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tgpbn" (UID: "51786990-2492-4142-994e-9e908bd478c1") : secret "insights-runtime-extractor-tls" not found Aug 04 10:28:26.003646 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:26.003565 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ttzdq_1955136f-4af9-4ab3-81df-790fa97cddab/serve-healthcheck-canary/0.log" Aug 04 10:28:27.678102 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:27.678017 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-787tx" event={"ID":"71172b55-f0af-407d-bb6f-89f2d92af16e","Type":"ContainerStarted","Data":"85a1863c89a961ea1778cadc57c01171f8b4643dfc17409724d8cfb48bd3aca8"} Aug 04 10:28:27.706962 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:27.705028 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-98c88fbb5-787tx" podStartSLOduration=1.313513564 podStartE2EDuration="3.705008039s" totalImagesPullingTime="2.391494475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:28:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:28:27.703709196 +0000 UTC m=+45.677791724" watchObservedRunningTime="2026-08-04 10:28:27.705008039 +0000 UTC m=+45.679090566" Aug 04 10:28:28.287046 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:28.287010 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:28.431757 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:28.431718 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:28.536137 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:28.536099 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:28.641348 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:28.641264 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:28.701881 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:28.701830 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:29.392219 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:29.392185 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:29.846800 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:29.846771 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/51786990-2492-4142-994e-9e908bd478c1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tgpbn\" (UID: \"51786990-2492-4142-994e-9e908bd478c1\") " pod="openshift-insights/insights-runtime-extractor-tgpbn" Aug 04 10:28:29.847185 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:29.846830 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/51786990-2492-4142-994e-9e908bd478c1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tgpbn\" (UID: \"51786990-2492-4142-994e-9e908bd478c1\") " pod="openshift-insights/insights-runtime-extractor-tgpbn" Aug 04 10:28:29.847185 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:29.846854 2535 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 10:28:29.847185 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:29.846911 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51786990-2492-4142-994e-9e908bd478c1-insights-runtime-extractor-tls podName:51786990-2492-4142-994e-9e908bd478c1 nodeName:}" failed. No retries permitted until 2026-08-04 10:28:37.846894539 +0000 UTC m=+55.820977044 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/51786990-2492-4142-994e-9e908bd478c1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tgpbn" (UID: "51786990-2492-4142-994e-9e908bd478c1") : secret "insights-runtime-extractor-tls" not found Aug 04 10:28:29.847185 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:29.846939 2535 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 10:28:29.847185 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:29.846995 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/51786990-2492-4142-994e-9e908bd478c1-kube-rbac-proxy-cm podName:51786990-2492-4142-994e-9e908bd478c1 nodeName:}" failed. No retries permitted until 2026-08-04 10:28:37.846977864 +0000 UTC m=+55.821060581 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/51786990-2492-4142-994e-9e908bd478c1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tgpbn" (UID: "51786990-2492-4142-994e-9e908bd478c1") : configmap "kube-rbac-proxy" not found Aug 04 10:28:29.992531 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:29.992501 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:30.505502 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:30.505461 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:30.606358 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:30.606324 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:30.789286 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:30.789257 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:31.187883 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:31.187801 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:31.910505 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:31.910460 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:32.011150 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:32.011121 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:32.111548 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:32.111516 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:32.212104 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:32.212030 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:32.311984 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:32.311958 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:32.591765 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:32.591728 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:32.769734 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:32.769701 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99-metrics-tls\") pod \"dns-default-7wdtv\" (UID: \"5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99\") " pod="openshift-dns/dns-default-7wdtv" Aug 04 10:28:32.771928 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:32.771899 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99-metrics-tls\") pod \"dns-default-7wdtv\" (UID: \"5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99\") " pod="openshift-dns/dns-default-7wdtv" Aug 04 10:28:32.782994 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:32.782966 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:32.808187 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:32.808164 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7wdtv" Aug 04 10:28:32.844764 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:32.844697 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:32.960187 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:32.960150 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7wdtv"] Aug 04 10:28:32.963727 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:28:32.963694 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fcf23dc_6a51_44e4_b4cc_dd2a77fe1c99.slice/crio-0c7426f9416f08a9f13bf159ecfeee0c25d9509a7d7d6ae29366dfd65258494d WatchSource:0}: Error finding container 0c7426f9416f08a9f13bf159ecfeee0c25d9509a7d7d6ae29366dfd65258494d: Status 404 returned error can't find the container with id 0c7426f9416f08a9f13bf159ecfeee0c25d9509a7d7d6ae29366dfd65258494d Aug 04 10:28:33.194928 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:33.194842 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:33.693045 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:33.693007 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7wdtv" event={"ID":"5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99","Type":"ContainerStarted","Data":"0c7426f9416f08a9f13bf159ecfeee0c25d9509a7d7d6ae29366dfd65258494d"} Aug 04 10:28:33.785784 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:33.785752 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:34.190285 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:34.190251 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:34.604301 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:34.604262 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:34.706114 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:34.706060 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:34.999363 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:34.999279 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:35.196619 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:35.196572 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:35.265652 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:35.265576 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:35.615612 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:35.615581 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:35.701525 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:35.701479 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7wdtv" event={"ID":"5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99","Type":"ContainerStarted","Data":"18ca689c2ec0374712da8e8e182732a9daef8759f293e6a6642ca2c6e828c455"} Aug 04 10:28:35.701679 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:35.701548 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7wdtv" event={"ID":"5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99","Type":"ContainerStarted","Data":"763b6749ddebe15a9d11a206ffcffa6ed302bf92c33ed22128c62b3fd724f420"} Aug 04 10:28:35.701679 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:35.701580 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7wdtv" Aug 04 10:28:35.715124 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:35.715100 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:35.725427 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:35.725384 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7wdtv" podStartSLOduration=17.375843138 podStartE2EDuration="19.725371196s" totalImagesPullingTime="2.349528058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:28:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:28:35.72505736 +0000 UTC m=+53.699139904" watchObservedRunningTime="2026-08-04 10:28:35.725371196 +0000 UTC m=+53.699453722" Aug 04 10:28:35.816690 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:35.816664 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:36.395323 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:36.392579 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:36.592550 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:36.592519 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:36.993516 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:36.993472 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:37.199017 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:37.198977 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:37.594249 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:37.594217 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:37.801213 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:37.801182 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:37.912319 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:37.912247 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/51786990-2492-4142-994e-9e908bd478c1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tgpbn\" (UID: \"51786990-2492-4142-994e-9e908bd478c1\") " pod="openshift-insights/insights-runtime-extractor-tgpbn" Aug 04 10:28:37.912319 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:37.912286 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/51786990-2492-4142-994e-9e908bd478c1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tgpbn\" (UID: \"51786990-2492-4142-994e-9e908bd478c1\") " pod="openshift-insights/insights-runtime-extractor-tgpbn" Aug 04 10:28:37.912628 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:37.912409 2535 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 10:28:37.912628 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:28:37.912506 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51786990-2492-4142-994e-9e908bd478c1-insights-runtime-extractor-tls podName:51786990-2492-4142-994e-9e908bd478c1 nodeName:}" failed. No retries permitted until 2026-08-04 10:28:53.912466572 +0000 UTC m=+71.886549094 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/51786990-2492-4142-994e-9e908bd478c1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tgpbn" (UID: "51786990-2492-4142-994e-9e908bd478c1") : secret "insights-runtime-extractor-tls" not found Aug 04 10:28:37.912822 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:37.912801 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/51786990-2492-4142-994e-9e908bd478c1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tgpbn\" (UID: \"51786990-2492-4142-994e-9e908bd478c1\") " pod="openshift-insights/insights-runtime-extractor-tgpbn" Aug 04 10:28:38.593151 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:38.593115 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:38.789387 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:38.789349 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:39.446480 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:39.446448 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:41.812864 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:41.812827 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:42.653530 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:42.653498 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lcxhd" Aug 04 10:28:45.706728 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:45.706694 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7wdtv" Aug 04 10:28:48.181633 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.181577 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73272ce5-1238-42c7-b697-29d21e373fc0-original-pull-secret\") pod \"global-pull-secret-syncer-zzlbv\" (UID: \"73272ce5-1238-42c7-b697-29d21e373fc0\") " pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:28:48.182092 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.181658 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5436bc5b-0740-4ccb-b4fd-857d4023d8e7-metrics-certs\") pod \"network-metrics-daemon-qgpx8\" (UID: \"5436bc5b-0740-4ccb-b4fd-857d4023d8e7\") " pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:28:48.183799 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.183775 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Aug 04 10:28:48.183867 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.183849 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Aug 04 10:28:48.194408 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.194381 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5436bc5b-0740-4ccb-b4fd-857d4023d8e7-metrics-certs\") pod \"network-metrics-daemon-qgpx8\" (UID: \"5436bc5b-0740-4ccb-b4fd-857d4023d8e7\") " pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:28:48.194522 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.194384 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73272ce5-1238-42c7-b697-29d21e373fc0-original-pull-secret\") pod \"global-pull-secret-syncer-zzlbv\" (UID: \"73272ce5-1238-42c7-b697-29d21e373fc0\") " pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:28:48.383471 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.383427 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w28sh\" (UniqueName: \"kubernetes.io/projected/b6f16f94-e1b9-4900-a75d-84badd1d9bb7-kube-api-access-w28sh\") pod \"network-check-target-2rc2x\" (UID: \"b6f16f94-e1b9-4900-a75d-84badd1d9bb7\") " pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:28:48.385802 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.385785 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Aug 04 10:28:48.386823 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.386809 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zzlbv" Aug 04 10:28:48.395613 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.395594 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-p48mn\"" Aug 04 10:28:48.396051 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.396028 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Aug 04 10:28:48.403562 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.403544 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qgpx8" Aug 04 10:28:48.407234 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.407210 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w28sh\" (UniqueName: \"kubernetes.io/projected/b6f16f94-e1b9-4900-a75d-84badd1d9bb7-kube-api-access-w28sh\") pod \"network-check-target-2rc2x\" (UID: \"b6f16f94-e1b9-4900-a75d-84badd1d9bb7\") " pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:28:48.487217 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.487016 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-clvkl\"" Aug 04 10:28:48.495010 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.494974 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:28:48.550959 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.550917 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zzlbv"] Aug 04 10:28:48.552597 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.552532 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qgpx8"] Aug 04 10:28:48.559115 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:28:48.557890 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73272ce5_1238_42c7_b697_29d21e373fc0.slice/crio-74759e8eb3521da900aa24ce54629efe72f51c352fe48ee2c9c64bf52e455ee3 WatchSource:0}: Error finding container 74759e8eb3521da900aa24ce54629efe72f51c352fe48ee2c9c64bf52e455ee3: Status 404 returned error can't find the container with id 74759e8eb3521da900aa24ce54629efe72f51c352fe48ee2c9c64bf52e455ee3 Aug 04 10:28:48.561040 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:28:48.561002 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5436bc5b_0740_4ccb_b4fd_857d4023d8e7.slice/crio-51517398db4b180b86cf26e3098f3a20de22f624d7c9bbca7791fd9147e804ce WatchSource:0}: Error finding container 51517398db4b180b86cf26e3098f3a20de22f624d7c9bbca7791fd9147e804ce: Status 404 returned error can't find the container with id 51517398db4b180b86cf26e3098f3a20de22f624d7c9bbca7791fd9147e804ce Aug 04 10:28:48.643054 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.643021 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2rc2x"] Aug 04 10:28:48.646636 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.646613 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-57966898bb-nvtr6"] Aug 04 10:28:48.647119 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:28:48.647092 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6f16f94_e1b9_4900_a75d_84badd1d9bb7.slice/crio-123e3f8897cbfab680a68f37014725b67a7acbb4c24b29608573ea161fde91cb WatchSource:0}: Error finding container 123e3f8897cbfab680a68f37014725b67a7acbb4c24b29608573ea161fde91cb: Status 404 returned error can't find the container with id 123e3f8897cbfab680a68f37014725b67a7acbb4c24b29608573ea161fde91cb Aug 04 10:28:48.651779 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.651760 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57966898bb-nvtr6" Aug 04 10:28:48.653702 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.653681 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Aug 04 10:28:48.653921 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.653903 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Aug 04 10:28:48.654024 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.654009 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Aug 04 10:28:48.654151 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.654130 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Aug 04 10:28:48.662242 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.662222 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-57966898bb-nvtr6"] Aug 04 10:28:48.686595 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.686530 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n69wp\" (UniqueName: \"kubernetes.io/projected/34ca2d46-56cd-46c9-8369-d754ae340ea1-kube-api-access-n69wp\") pod \"klusterlet-addon-workmgr-57966898bb-nvtr6\" (UID: \"34ca2d46-56cd-46c9-8369-d754ae340ea1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57966898bb-nvtr6" Aug 04 10:28:48.686595 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.686571 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/34ca2d46-56cd-46c9-8369-d754ae340ea1-klusterlet-config\") pod \"klusterlet-addon-workmgr-57966898bb-nvtr6\" (UID: \"34ca2d46-56cd-46c9-8369-d754ae340ea1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57966898bb-nvtr6" Aug 04 10:28:48.686737 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.686597 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/34ca2d46-56cd-46c9-8369-d754ae340ea1-tmp\") pod \"klusterlet-addon-workmgr-57966898bb-nvtr6\" (UID: \"34ca2d46-56cd-46c9-8369-d754ae340ea1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57966898bb-nvtr6" Aug 04 10:28:48.738156 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.738120 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qgpx8" event={"ID":"5436bc5b-0740-4ccb-b4fd-857d4023d8e7","Type":"ContainerStarted","Data":"51517398db4b180b86cf26e3098f3a20de22f624d7c9bbca7791fd9147e804ce"} Aug 04 10:28:48.739293 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.739269 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zzlbv" event={"ID":"73272ce5-1238-42c7-b697-29d21e373fc0","Type":"ContainerStarted","Data":"74759e8eb3521da900aa24ce54629efe72f51c352fe48ee2c9c64bf52e455ee3"} Aug 04 10:28:48.740238 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.740216 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2rc2x" event={"ID":"b6f16f94-e1b9-4900-a75d-84badd1d9bb7","Type":"ContainerStarted","Data":"123e3f8897cbfab680a68f37014725b67a7acbb4c24b29608573ea161fde91cb"} Aug 04 10:28:48.755887 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.755867 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-74d45d67dc-rb8sl"] Aug 04 10:28:48.759297 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.759280 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" Aug 04 10:28:48.762709 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.762693 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Aug 04 10:28:48.763062 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.763048 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Aug 04 10:28:48.763313 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.763296 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Aug 04 10:28:48.763381 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.763344 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wt4lh\"" Aug 04 10:28:48.765861 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.765815 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:48.772639 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.772615 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Aug 04 10:28:48.777192 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.777172 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74d45d67dc-rb8sl"] Aug 04 10:28:48.786890 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.786870 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/34ca2d46-56cd-46c9-8369-d754ae340ea1-klusterlet-config\") pod \"klusterlet-addon-workmgr-57966898bb-nvtr6\" (UID: \"34ca2d46-56cd-46c9-8369-d754ae340ea1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57966898bb-nvtr6" Aug 04 10:28:48.786969 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.786898 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/34ca2d46-56cd-46c9-8369-d754ae340ea1-tmp\") pod \"klusterlet-addon-workmgr-57966898bb-nvtr6\" (UID: \"34ca2d46-56cd-46c9-8369-d754ae340ea1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57966898bb-nvtr6" Aug 04 10:28:48.786969 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.786948 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n69wp\" (UniqueName: \"kubernetes.io/projected/34ca2d46-56cd-46c9-8369-d754ae340ea1-kube-api-access-n69wp\") pod \"klusterlet-addon-workmgr-57966898bb-nvtr6\" (UID: \"34ca2d46-56cd-46c9-8369-d754ae340ea1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57966898bb-nvtr6" Aug 04 10:28:48.787296 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.787277 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/34ca2d46-56cd-46c9-8369-d754ae340ea1-tmp\") pod \"klusterlet-addon-workmgr-57966898bb-nvtr6\" (UID: \"34ca2d46-56cd-46c9-8369-d754ae340ea1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57966898bb-nvtr6" Aug 04 10:28:48.789176 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.789159 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/34ca2d46-56cd-46c9-8369-d754ae340ea1-klusterlet-config\") pod \"klusterlet-addon-workmgr-57966898bb-nvtr6\" (UID: \"34ca2d46-56cd-46c9-8369-d754ae340ea1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57966898bb-nvtr6" Aug 04 10:28:48.797770 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.797752 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n69wp\" (UniqueName: \"kubernetes.io/projected/34ca2d46-56cd-46c9-8369-d754ae340ea1-kube-api-access-n69wp\") pod \"klusterlet-addon-workmgr-57966898bb-nvtr6\" (UID: \"34ca2d46-56cd-46c9-8369-d754ae340ea1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57966898bb-nvtr6" Aug 04 10:28:48.842726 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.842705 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:48.888041 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.888018 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7a214a2a-3267-48bb-8cd4-03f8fcf5bce6-registry-tls\") pod \"image-registry-74d45d67dc-rb8sl\" (UID: \"7a214a2a-3267-48bb-8cd4-03f8fcf5bce6\") " pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" Aug 04 10:28:48.888136 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.888050 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7a214a2a-3267-48bb-8cd4-03f8fcf5bce6-installation-pull-secrets\") pod \"image-registry-74d45d67dc-rb8sl\" (UID: \"7a214a2a-3267-48bb-8cd4-03f8fcf5bce6\") " pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" Aug 04 10:28:48.888136 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.888073 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znmwd\" (UniqueName: \"kubernetes.io/projected/7a214a2a-3267-48bb-8cd4-03f8fcf5bce6-kube-api-access-znmwd\") pod \"image-registry-74d45d67dc-rb8sl\" (UID: \"7a214a2a-3267-48bb-8cd4-03f8fcf5bce6\") " pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" Aug 04 10:28:48.888251 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.888172 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7a214a2a-3267-48bb-8cd4-03f8fcf5bce6-registry-certificates\") pod \"image-registry-74d45d67dc-rb8sl\" (UID: \"7a214a2a-3267-48bb-8cd4-03f8fcf5bce6\") " pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" Aug 04 10:28:48.888251 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.888207 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7a214a2a-3267-48bb-8cd4-03f8fcf5bce6-ca-trust-extracted\") pod \"image-registry-74d45d67dc-rb8sl\" (UID: \"7a214a2a-3267-48bb-8cd4-03f8fcf5bce6\") " pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" Aug 04 10:28:48.888251 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.888237 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7a214a2a-3267-48bb-8cd4-03f8fcf5bce6-image-registry-private-configuration\") pod \"image-registry-74d45d67dc-rb8sl\" (UID: \"7a214a2a-3267-48bb-8cd4-03f8fcf5bce6\") " pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" Aug 04 10:28:48.888393 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.888263 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7a214a2a-3267-48bb-8cd4-03f8fcf5bce6-trusted-ca\") pod \"image-registry-74d45d67dc-rb8sl\" (UID: \"7a214a2a-3267-48bb-8cd4-03f8fcf5bce6\") " pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" Aug 04 10:28:48.888393 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.888298 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7a214a2a-3267-48bb-8cd4-03f8fcf5bce6-bound-sa-token\") pod \"image-registry-74d45d67dc-rb8sl\" (UID: \"7a214a2a-3267-48bb-8cd4-03f8fcf5bce6\") " pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" Aug 04 10:28:48.961615 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.961556 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57966898bb-nvtr6" Aug 04 10:28:48.989249 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.989225 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7a214a2a-3267-48bb-8cd4-03f8fcf5bce6-registry-tls\") pod \"image-registry-74d45d67dc-rb8sl\" (UID: \"7a214a2a-3267-48bb-8cd4-03f8fcf5bce6\") " pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" Aug 04 10:28:48.989349 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.989265 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7a214a2a-3267-48bb-8cd4-03f8fcf5bce6-installation-pull-secrets\") pod \"image-registry-74d45d67dc-rb8sl\" (UID: \"7a214a2a-3267-48bb-8cd4-03f8fcf5bce6\") " pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" Aug 04 10:28:48.989349 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.989289 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znmwd\" (UniqueName: \"kubernetes.io/projected/7a214a2a-3267-48bb-8cd4-03f8fcf5bce6-kube-api-access-znmwd\") pod \"image-registry-74d45d67dc-rb8sl\" (UID: \"7a214a2a-3267-48bb-8cd4-03f8fcf5bce6\") " pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" Aug 04 10:28:48.989524 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.989354 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7a214a2a-3267-48bb-8cd4-03f8fcf5bce6-registry-certificates\") pod \"image-registry-74d45d67dc-rb8sl\" (UID: \"7a214a2a-3267-48bb-8cd4-03f8fcf5bce6\") " pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" Aug 04 10:28:48.989524 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.989381 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7a214a2a-3267-48bb-8cd4-03f8fcf5bce6-ca-trust-extracted\") pod \"image-registry-74d45d67dc-rb8sl\" (UID: \"7a214a2a-3267-48bb-8cd4-03f8fcf5bce6\") " pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" Aug 04 10:28:48.989524 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.989411 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7a214a2a-3267-48bb-8cd4-03f8fcf5bce6-image-registry-private-configuration\") pod \"image-registry-74d45d67dc-rb8sl\" (UID: \"7a214a2a-3267-48bb-8cd4-03f8fcf5bce6\") " pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" Aug 04 10:28:48.989524 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.989434 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7a214a2a-3267-48bb-8cd4-03f8fcf5bce6-trusted-ca\") pod \"image-registry-74d45d67dc-rb8sl\" (UID: \"7a214a2a-3267-48bb-8cd4-03f8fcf5bce6\") " pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" Aug 04 10:28:48.989524 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.989471 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7a214a2a-3267-48bb-8cd4-03f8fcf5bce6-bound-sa-token\") pod \"image-registry-74d45d67dc-rb8sl\" (UID: \"7a214a2a-3267-48bb-8cd4-03f8fcf5bce6\") " pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" Aug 04 10:28:48.990081 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.990063 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7a214a2a-3267-48bb-8cd4-03f8fcf5bce6-ca-trust-extracted\") pod \"image-registry-74d45d67dc-rb8sl\" (UID: \"7a214a2a-3267-48bb-8cd4-03f8fcf5bce6\") " pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" Aug 04 10:28:48.991997 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.991104 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7a214a2a-3267-48bb-8cd4-03f8fcf5bce6-registry-certificates\") pod \"image-registry-74d45d67dc-rb8sl\" (UID: \"7a214a2a-3267-48bb-8cd4-03f8fcf5bce6\") " pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" Aug 04 10:28:48.991997 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.991930 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7a214a2a-3267-48bb-8cd4-03f8fcf5bce6-trusted-ca\") pod \"image-registry-74d45d67dc-rb8sl\" (UID: \"7a214a2a-3267-48bb-8cd4-03f8fcf5bce6\") " pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" Aug 04 10:28:48.992388 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.992364 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7a214a2a-3267-48bb-8cd4-03f8fcf5bce6-image-registry-private-configuration\") pod \"image-registry-74d45d67dc-rb8sl\" (UID: \"7a214a2a-3267-48bb-8cd4-03f8fcf5bce6\") " pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" Aug 04 10:28:48.992579 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.992553 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7a214a2a-3267-48bb-8cd4-03f8fcf5bce6-installation-pull-secrets\") pod \"image-registry-74d45d67dc-rb8sl\" (UID: \"7a214a2a-3267-48bb-8cd4-03f8fcf5bce6\") " pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" Aug 04 10:28:48.993219 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.993201 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7a214a2a-3267-48bb-8cd4-03f8fcf5bce6-registry-tls\") pod \"image-registry-74d45d67dc-rb8sl\" (UID: \"7a214a2a-3267-48bb-8cd4-03f8fcf5bce6\") " pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" Aug 04 10:28:48.998162 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.997910 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znmwd\" (UniqueName: \"kubernetes.io/projected/7a214a2a-3267-48bb-8cd4-03f8fcf5bce6-kube-api-access-znmwd\") pod \"image-registry-74d45d67dc-rb8sl\" (UID: \"7a214a2a-3267-48bb-8cd4-03f8fcf5bce6\") " pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" Aug 04 10:28:48.998162 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:48.998074 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7a214a2a-3267-48bb-8cd4-03f8fcf5bce6-bound-sa-token\") pod \"image-registry-74d45d67dc-rb8sl\" (UID: \"7a214a2a-3267-48bb-8cd4-03f8fcf5bce6\") " pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" Aug 04 10:28:49.067948 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:49.067906 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" Aug 04 10:28:49.125530 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:49.125444 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-57966898bb-nvtr6"] Aug 04 10:28:49.129436 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:28:49.129406 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34ca2d46_56cd_46c9_8369_d754ae340ea1.slice/crio-55c623847ab5e719069633dd4c59d9d75da6954c6e8d45512efb43adc5d543b7 WatchSource:0}: Error finding container 55c623847ab5e719069633dd4c59d9d75da6954c6e8d45512efb43adc5d543b7: Status 404 returned error can't find the container with id 55c623847ab5e719069633dd4c59d9d75da6954c6e8d45512efb43adc5d543b7 Aug 04 10:28:49.214935 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:49.214865 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:49.254122 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:49.254050 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74d45d67dc-rb8sl"] Aug 04 10:28:49.259172 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:28:49.259141 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a214a2a_3267_48bb_8cd4_03f8fcf5bce6.slice/crio-629eb712b5ed6a8cbe20969c47d108079b6e60429b3eca022453137a20dec354 WatchSource:0}: Error finding container 629eb712b5ed6a8cbe20969c47d108079b6e60429b3eca022453137a20dec354: Status 404 returned error can't find the container with id 629eb712b5ed6a8cbe20969c47d108079b6e60429b3eca022453137a20dec354 Aug 04 10:28:49.317291 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:49.317261 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:49.735866 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:49.735798 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:49.746501 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:49.746347 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" event={"ID":"7a214a2a-3267-48bb-8cd4-03f8fcf5bce6","Type":"ContainerStarted","Data":"75a77e8764ede6c57704cda5da33e11110e13651c62096eea94437c0b73676db"} Aug 04 10:28:49.746501 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:49.746404 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" event={"ID":"7a214a2a-3267-48bb-8cd4-03f8fcf5bce6","Type":"ContainerStarted","Data":"629eb712b5ed6a8cbe20969c47d108079b6e60429b3eca022453137a20dec354"} Aug 04 10:28:49.746501 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:49.746449 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" Aug 04 10:28:49.751358 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:49.751305 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57966898bb-nvtr6" event={"ID":"34ca2d46-56cd-46c9-8369-d754ae340ea1","Type":"ContainerStarted","Data":"55c623847ab5e719069633dd4c59d9d75da6954c6e8d45512efb43adc5d543b7"} Aug 04 10:28:49.776953 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:49.776870 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" podStartSLOduration=1.776848314 podStartE2EDuration="1.776848314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:28:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:28:49.774242674 +0000 UTC m=+67.748325250" watchObservedRunningTime="2026-08-04 10:28:49.776848314 +0000 UTC m=+67.750930842" Aug 04 10:28:49.829871 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:49.829830 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:49.934421 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:49.934381 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-vrblv"] Aug 04 10:28:49.961208 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:49.961177 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-vrblv"] Aug 04 10:28:49.961383 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:49.961299 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-vrblv" Aug 04 10:28:49.964442 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:49.963808 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Aug 04 10:28:49.964442 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:49.964096 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-vl2zv\"" Aug 04 10:28:49.999662 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:49.999601 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0c82316b-a816-4c9e-b307-686d0e17e679-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-vrblv\" (UID: \"0c82316b-a816-4c9e-b307-686d0e17e679\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-vrblv" Aug 04 10:28:50.100210 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:50.100161 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0c82316b-a816-4c9e-b307-686d0e17e679-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-vrblv\" (UID: \"0c82316b-a816-4c9e-b307-686d0e17e679\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-vrblv" Aug 04 10:28:50.111444 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:50.111416 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0c82316b-a816-4c9e-b307-686d0e17e679-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-vrblv\" (UID: \"0c82316b-a816-4c9e-b307-686d0e17e679\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-vrblv" Aug 04 10:28:50.161583 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:50.161514 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:50.240767 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:50.240706 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:50.275028 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:50.275001 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-vrblv" Aug 04 10:28:50.464262 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:50.464195 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-vrblv"] Aug 04 10:28:50.473265 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:28:50.473230 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c82316b_a816_4c9e_b307_686d0e17e679.slice/crio-a421101dae071bbd85560a84d13a565c32ac897c7fa6a3a3dfd05783a98e7aae WatchSource:0}: Error finding container a421101dae071bbd85560a84d13a565c32ac897c7fa6a3a3dfd05783a98e7aae: Status 404 returned error can't find the container with id a421101dae071bbd85560a84d13a565c32ac897c7fa6a3a3dfd05783a98e7aae Aug 04 10:28:50.491915 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:50.491866 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:50.592890 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:50.592742 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:50.760139 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:50.760094 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qgpx8" event={"ID":"5436bc5b-0740-4ccb-b4fd-857d4023d8e7","Type":"ContainerStarted","Data":"fcc3204c172f395291dd44251c8f6fa73580e114f313a0f4c21b318abd3440c0"} Aug 04 10:28:50.760320 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:50.760166 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qgpx8" event={"ID":"5436bc5b-0740-4ccb-b4fd-857d4023d8e7","Type":"ContainerStarted","Data":"e8d6fbd831bbf30fd3bb6ee37fd970bb0df259164a2ed74a32c6afe47b9c1f1f"} Aug 04 10:28:50.762055 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:50.762027 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-vrblv" event={"ID":"0c82316b-a816-4c9e-b307-686d0e17e679","Type":"ContainerStarted","Data":"a421101dae071bbd85560a84d13a565c32ac897c7fa6a3a3dfd05783a98e7aae"} Aug 04 10:28:50.782035 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:50.781419 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qgpx8" podStartSLOduration=67.232118293 podStartE2EDuration="1m8.781398869s" totalImagesPullingTime="1.549280576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:28:50.779544639 +0000 UTC m=+68.753627186" watchObservedRunningTime="2026-08-04 10:28:50.781398869 +0000 UTC m=+68.755481397" Aug 04 10:28:51.499775 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:51.499687 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:51.897958 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:51.897914 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:52.035810 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:52.035777 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:53.931334 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:53.931302 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/51786990-2492-4142-994e-9e908bd478c1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tgpbn\" (UID: \"51786990-2492-4142-994e-9e908bd478c1\") " pod="openshift-insights/insights-runtime-extractor-tgpbn" Aug 04 10:28:53.933828 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:53.933803 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/51786990-2492-4142-994e-9e908bd478c1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tgpbn\" (UID: \"51786990-2492-4142-994e-9e908bd478c1\") " pod="openshift-insights/insights-runtime-extractor-tgpbn" Aug 04 10:28:54.199071 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:54.198989 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ntlkp\"" Aug 04 10:28:54.206945 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:54.206916 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tgpbn" Aug 04 10:28:56.124365 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:56.124325 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tgpbn"] Aug 04 10:28:56.128586 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:28:56.128554 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51786990_2492_4142_994e_9e908bd478c1.slice/crio-2b8d4853bf7c71fece14e74945d3daed2b90f2435a3cec976b00d0d35ca84487 WatchSource:0}: Error finding container 2b8d4853bf7c71fece14e74945d3daed2b90f2435a3cec976b00d0d35ca84487: Status 404 returned error can't find the container with id 2b8d4853bf7c71fece14e74945d3daed2b90f2435a3cec976b00d0d35ca84487 Aug 04 10:28:56.782361 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:56.782332 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:56.784931 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:56.784899 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2rc2x" event={"ID":"b6f16f94-e1b9-4900-a75d-84badd1d9bb7","Type":"ContainerStarted","Data":"79a0b99fed2b22b344f7aab408c37080590596e2e9b7db3351eea84bcc3713f2"} Aug 04 10:28:56.785077 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:56.785000 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:28:56.786600 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:56.786574 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tgpbn" event={"ID":"51786990-2492-4142-994e-9e908bd478c1","Type":"ContainerStarted","Data":"a99467ccee46efdc964a6327ba7860b5f88f845d5a3d06b9d720d16cd0e5cc8e"} Aug 04 10:28:56.786690 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:56.786621 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tgpbn" event={"ID":"51786990-2492-4142-994e-9e908bd478c1","Type":"ContainerStarted","Data":"2b8d4853bf7c71fece14e74945d3daed2b90f2435a3cec976b00d0d35ca84487"} Aug 04 10:28:56.788128 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:56.788097 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zzlbv" event={"ID":"73272ce5-1238-42c7-b697-29d21e373fc0","Type":"ContainerStarted","Data":"12b6cab159da989d97ab70e9e249ba3aa26c57ea8304aac5187fad8e6d0c3c32"} Aug 04 10:28:56.789518 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:56.789480 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-vrblv" event={"ID":"0c82316b-a816-4c9e-b307-686d0e17e679","Type":"ContainerStarted","Data":"94213cb3cde48d03a18313f64edd559b53f96f1beaee471dea5e4f64eb9f7482"} Aug 04 10:28:56.789598 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:56.789536 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-vrblv" Aug 04 10:28:56.790927 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:56.790902 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57966898bb-nvtr6" event={"ID":"34ca2d46-56cd-46c9-8369-d754ae340ea1","Type":"ContainerStarted","Data":"5bcac736cd85f7c55ae6292ed92b8f5c0f8fe2ffea91ac2a1a2c1f120c01a743"} Aug 04 10:28:56.791120 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:56.791103 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57966898bb-nvtr6" Aug 04 10:28:56.792590 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:56.792572 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57966898bb-nvtr6" Aug 04 10:28:56.794750 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:56.794733 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-vrblv" Aug 04 10:28:56.804229 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:56.804192 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2rc2x" podStartSLOduration=67.48542466 podStartE2EDuration="1m14.804178973s" totalImagesPullingTime="7.318754313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:28:56.803466385 +0000 UTC m=+74.777548911" watchObservedRunningTime="2026-08-04 10:28:56.804178973 +0000 UTC m=+74.778261500" Aug 04 10:28:56.822613 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:56.822571 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-vrblv" podStartSLOduration=2.331187211 podStartE2EDuration="7.822555719s" totalImagesPullingTime="5.491368508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:28:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:28:56.821243145 +0000 UTC m=+74.795325685" watchObservedRunningTime="2026-08-04 10:28:56.822555719 +0000 UTC m=+74.796638246" Aug 04 10:28:56.837983 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:56.837925 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57966898bb-nvtr6" podStartSLOduration=1.980280594 podStartE2EDuration="8.837904401s" totalImagesPullingTime="6.857623807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:28:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:28:56.835804889 +0000 UTC m=+74.809887410" watchObservedRunningTime="2026-08-04 10:28:56.837904401 +0000 UTC m=+74.811986930" Aug 04 10:28:56.844753 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:56.844723 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:56.856046 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:56.855993 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zzlbv" podStartSLOduration=67.43792827 podStartE2EDuration="1m14.855974398s" totalImagesPullingTime="7.418046128s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 10:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:28:56.854076003 +0000 UTC m=+74.828158547" watchObservedRunningTime="2026-08-04 10:28:56.855974398 +0000 UTC m=+74.830056927" Aug 04 10:28:56.962624 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:56.962585 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:57.015975 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:57.015950 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-xlkqn"] Aug 04 10:28:57.018824 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:57.018806 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-758d7cb874-xlkqn" Aug 04 10:28:57.021098 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:57.021072 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Aug 04 10:28:57.021206 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:57.021100 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-mpq2r\"" Aug 04 10:28:57.021206 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:57.021122 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Aug 04 10:28:57.021206 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:57.021126 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Aug 04 10:28:57.021206 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:57.021172 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Aug 04 10:28:57.021455 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:57.021075 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Aug 04 10:28:57.031747 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:57.031725 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-xlkqn"] Aug 04 10:28:57.038367 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:57.038323 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:57.148690 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:57.148665 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:57.154728 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:57.154697 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c6ade897-5b7d-49bf-a4ef-e01062cadf09-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-xlkqn\" (UID: \"c6ade897-5b7d-49bf-a4ef-e01062cadf09\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-xlkqn" Aug 04 10:28:57.154848 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:57.154736 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwkkg\" (UniqueName: \"kubernetes.io/projected/c6ade897-5b7d-49bf-a4ef-e01062cadf09-kube-api-access-kwkkg\") pod \"prometheus-operator-758d7cb874-xlkqn\" (UID: \"c6ade897-5b7d-49bf-a4ef-e01062cadf09\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-xlkqn" Aug 04 10:28:57.154848 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:57.154829 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c6ade897-5b7d-49bf-a4ef-e01062cadf09-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-xlkqn\" (UID: \"c6ade897-5b7d-49bf-a4ef-e01062cadf09\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-xlkqn" Aug 04 10:28:57.154955 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:57.154873 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6ade897-5b7d-49bf-a4ef-e01062cadf09-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-xlkqn\" (UID: \"c6ade897-5b7d-49bf-a4ef-e01062cadf09\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-xlkqn" Aug 04 10:28:57.255632 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:57.255594 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c6ade897-5b7d-49bf-a4ef-e01062cadf09-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-xlkqn\" (UID: \"c6ade897-5b7d-49bf-a4ef-e01062cadf09\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-xlkqn" Aug 04 10:28:57.255632 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:57.255637 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwkkg\" (UniqueName: \"kubernetes.io/projected/c6ade897-5b7d-49bf-a4ef-e01062cadf09-kube-api-access-kwkkg\") pod \"prometheus-operator-758d7cb874-xlkqn\" (UID: \"c6ade897-5b7d-49bf-a4ef-e01062cadf09\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-xlkqn" Aug 04 10:28:57.255777 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:57.255758 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c6ade897-5b7d-49bf-a4ef-e01062cadf09-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-xlkqn\" (UID: \"c6ade897-5b7d-49bf-a4ef-e01062cadf09\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-xlkqn" Aug 04 10:28:57.255823 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:57.255809 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6ade897-5b7d-49bf-a4ef-e01062cadf09-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-xlkqn\" (UID: \"c6ade897-5b7d-49bf-a4ef-e01062cadf09\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-xlkqn" Aug 04 10:28:57.256373 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:57.256354 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6ade897-5b7d-49bf-a4ef-e01062cadf09-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-xlkqn\" (UID: \"c6ade897-5b7d-49bf-a4ef-e01062cadf09\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-xlkqn" Aug 04 10:28:57.257863 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:57.257844 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c6ade897-5b7d-49bf-a4ef-e01062cadf09-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-xlkqn\" (UID: \"c6ade897-5b7d-49bf-a4ef-e01062cadf09\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-xlkqn" Aug 04 10:28:57.258100 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:57.258080 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c6ade897-5b7d-49bf-a4ef-e01062cadf09-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-xlkqn\" (UID: \"c6ade897-5b7d-49bf-a4ef-e01062cadf09\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-xlkqn" Aug 04 10:28:57.264083 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:57.264061 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwkkg\" (UniqueName: \"kubernetes.io/projected/c6ade897-5b7d-49bf-a4ef-e01062cadf09-kube-api-access-kwkkg\") pod \"prometheus-operator-758d7cb874-xlkqn\" (UID: \"c6ade897-5b7d-49bf-a4ef-e01062cadf09\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-xlkqn" Aug 04 10:28:57.328205 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:57.328149 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-758d7cb874-xlkqn" Aug 04 10:28:57.453925 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:57.453883 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-xlkqn"] Aug 04 10:28:57.458499 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:28:57.458455 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6ade897_5b7d_49bf_a4ef_e01062cadf09.slice/crio-5ddb8b1ba72728cda81bfc20b53ddc502a2bd05801f3b1d391ed6c0d241e1f2f WatchSource:0}: Error finding container 5ddb8b1ba72728cda81bfc20b53ddc502a2bd05801f3b1d391ed6c0d241e1f2f: Status 404 returned error can't find the container with id 5ddb8b1ba72728cda81bfc20b53ddc502a2bd05801f3b1d391ed6c0d241e1f2f Aug 04 10:28:57.610368 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:57.610284 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:57.703758 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:57.703725 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:57.796994 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:57.796954 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tgpbn" event={"ID":"51786990-2492-4142-994e-9e908bd478c1","Type":"ContainerStarted","Data":"65d45239f984c9049e753683517a6388238348be19caf1ccf9251fcb1cb0bc03"} Aug 04 10:28:57.798447 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:57.798413 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-xlkqn" event={"ID":"c6ade897-5b7d-49bf-a4ef-e01062cadf09","Type":"ContainerStarted","Data":"5ddb8b1ba72728cda81bfc20b53ddc502a2bd05801f3b1d391ed6c0d241e1f2f"} Aug 04 10:28:58.806665 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:58.806326 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tgpbn" event={"ID":"51786990-2492-4142-994e-9e908bd478c1","Type":"ContainerStarted","Data":"a721265a8fe179f6967011ef3eaae88b1da889588262ce2dac253f6ca943842e"} Aug 04 10:28:58.827328 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:58.827275 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-tgpbn" podStartSLOduration=34.839212797 podStartE2EDuration="36.827254519s" totalImagesPullingTime="1.988041722s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 10:28:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:28:58.825878849 +0000 UTC m=+76.799961399" watchObservedRunningTime="2026-08-04 10:28:58.827254519 +0000 UTC m=+76.801337050" Aug 04 10:28:59.365626 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:59.365552 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:59.814328 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:59.814288 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-xlkqn" event={"ID":"c6ade897-5b7d-49bf-a4ef-e01062cadf09","Type":"ContainerStarted","Data":"1ad271f3521b293b5d400031697daa622866d4451569c3e59a581406393fb975"} Aug 04 10:28:59.814829 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:59.814377 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-xlkqn" event={"ID":"c6ade897-5b7d-49bf-a4ef-e01062cadf09","Type":"ContainerStarted","Data":"30694087dadeccd9e23e44f09035b17f78435deaa70d754c5857b6a7e500910a"} Aug 04 10:28:59.842947 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:28:59.842892 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-758d7cb874-xlkqn" podStartSLOduration=2.261637475 podStartE2EDuration="3.84287437s" totalImagesPullingTime="1.581236895s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:28:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:28:59.84188447 +0000 UTC m=+77.815967008" watchObservedRunningTime="2026-08-04 10:28:59.84287437 +0000 UTC m=+77.816956915" Aug 04 10:29:01.123228 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.123192 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:01.395505 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.395396 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-9d8mh"] Aug 04 10:29:01.398474 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.398446 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:01.399404 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.399385 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-w798t"] Aug 04 10:29:01.399535 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.399518 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-9d8mh" Aug 04 10:29:01.401635 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.401449 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Aug 04 10:29:01.401635 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.401478 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Aug 04 10:29:01.403123 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.402637 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-w798t" Aug 04 10:29:01.403123 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.402824 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Aug 04 10:29:01.403123 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.403080 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-8tpww\"" Aug 04 10:29:01.404370 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.404348 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Aug 04 10:29:01.404599 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.404583 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Aug 04 10:29:01.404743 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.404592 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Aug 04 10:29:01.404907 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.404593 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rg4dt\"" Aug 04 10:29:01.414980 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.414955 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-9d8mh"] Aug 04 10:29:01.482240 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.482212 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:01.487302 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.487275 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsl57\" (UniqueName: \"kubernetes.io/projected/4fda4dea-ac20-4e18-8861-80a87970ca59-kube-api-access-qsl57\") pod \"node-exporter-w798t\" (UID: \"4fda4dea-ac20-4e18-8861-80a87970ca59\") " pod="openshift-monitoring/node-exporter-w798t" Aug 04 10:29:01.487416 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.487315 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4fda4dea-ac20-4e18-8861-80a87970ca59-node-exporter-textfile\") pod \"node-exporter-w798t\" (UID: \"4fda4dea-ac20-4e18-8861-80a87970ca59\") " pod="openshift-monitoring/node-exporter-w798t" Aug 04 10:29:01.487416 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.487358 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4fda4dea-ac20-4e18-8861-80a87970ca59-node-exporter-tls\") pod \"node-exporter-w798t\" (UID: \"4fda4dea-ac20-4e18-8861-80a87970ca59\") " pod="openshift-monitoring/node-exporter-w798t" Aug 04 10:29:01.487416 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.487392 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c0c22b1-8555-4ff4-9ead-afe3b4a58817-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-9d8mh\" (UID: \"3c0c22b1-8555-4ff4-9ead-afe3b4a58817\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-9d8mh" Aug 04 10:29:01.487584 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.487421 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4fda4dea-ac20-4e18-8861-80a87970ca59-root\") pod \"node-exporter-w798t\" (UID: \"4fda4dea-ac20-4e18-8861-80a87970ca59\") " pod="openshift-monitoring/node-exporter-w798t" Aug 04 10:29:01.487584 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.487457 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3c0c22b1-8555-4ff4-9ead-afe3b4a58817-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-9d8mh\" (UID: \"3c0c22b1-8555-4ff4-9ead-afe3b4a58817\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-9d8mh" Aug 04 10:29:01.487584 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.487519 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3c0c22b1-8555-4ff4-9ead-afe3b4a58817-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-9d8mh\" (UID: \"3c0c22b1-8555-4ff4-9ead-afe3b4a58817\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-9d8mh" Aug 04 10:29:01.487696 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.487581 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4fda4dea-ac20-4e18-8861-80a87970ca59-node-exporter-wtmp\") pod \"node-exporter-w798t\" (UID: \"4fda4dea-ac20-4e18-8861-80a87970ca59\") " pod="openshift-monitoring/node-exporter-w798t" Aug 04 10:29:01.487696 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.487620 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4fda4dea-ac20-4e18-8861-80a87970ca59-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w798t\" (UID: \"4fda4dea-ac20-4e18-8861-80a87970ca59\") " pod="openshift-monitoring/node-exporter-w798t" Aug 04 10:29:01.487696 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.487650 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3c0c22b1-8555-4ff4-9ead-afe3b4a58817-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-9d8mh\" (UID: \"3c0c22b1-8555-4ff4-9ead-afe3b4a58817\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-9d8mh" Aug 04 10:29:01.487696 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.487683 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4fda4dea-ac20-4e18-8861-80a87970ca59-node-exporter-accelerators-collector-config\") pod \"node-exporter-w798t\" (UID: \"4fda4dea-ac20-4e18-8861-80a87970ca59\") " pod="openshift-monitoring/node-exporter-w798t" Aug 04 10:29:01.487871 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.487706 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c0c22b1-8555-4ff4-9ead-afe3b4a58817-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-9d8mh\" (UID: \"3c0c22b1-8555-4ff4-9ead-afe3b4a58817\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-9d8mh" Aug 04 10:29:01.487871 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.487756 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4fda4dea-ac20-4e18-8861-80a87970ca59-metrics-client-ca\") pod \"node-exporter-w798t\" (UID: \"4fda4dea-ac20-4e18-8861-80a87970ca59\") " pod="openshift-monitoring/node-exporter-w798t" Aug 04 10:29:01.487871 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.487786 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxqsg\" (UniqueName: \"kubernetes.io/projected/3c0c22b1-8555-4ff4-9ead-afe3b4a58817-kube-api-access-sxqsg\") pod \"kube-state-metrics-6bd9664c4-9d8mh\" (UID: \"3c0c22b1-8555-4ff4-9ead-afe3b4a58817\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-9d8mh" Aug 04 10:29:01.487871 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.487835 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fda4dea-ac20-4e18-8861-80a87970ca59-sys\") pod \"node-exporter-w798t\" (UID: \"4fda4dea-ac20-4e18-8861-80a87970ca59\") " pod="openshift-monitoring/node-exporter-w798t" Aug 04 10:29:01.573087 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.573054 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:01.589188 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.589156 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4fda4dea-ac20-4e18-8861-80a87970ca59-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w798t\" (UID: \"4fda4dea-ac20-4e18-8861-80a87970ca59\") " pod="openshift-monitoring/node-exporter-w798t" Aug 04 10:29:01.589319 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.589202 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3c0c22b1-8555-4ff4-9ead-afe3b4a58817-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-9d8mh\" (UID: \"3c0c22b1-8555-4ff4-9ead-afe3b4a58817\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-9d8mh" Aug 04 10:29:01.589319 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.589234 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4fda4dea-ac20-4e18-8861-80a87970ca59-node-exporter-accelerators-collector-config\") pod \"node-exporter-w798t\" (UID: \"4fda4dea-ac20-4e18-8861-80a87970ca59\") " pod="openshift-monitoring/node-exporter-w798t" Aug 04 10:29:01.589319 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.589258 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c0c22b1-8555-4ff4-9ead-afe3b4a58817-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-9d8mh\" (UID: \"3c0c22b1-8555-4ff4-9ead-afe3b4a58817\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-9d8mh" Aug 04 10:29:01.589319 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.589288 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4fda4dea-ac20-4e18-8861-80a87970ca59-metrics-client-ca\") pod \"node-exporter-w798t\" (UID: \"4fda4dea-ac20-4e18-8861-80a87970ca59\") " pod="openshift-monitoring/node-exporter-w798t" Aug 04 10:29:01.589319 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.589314 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxqsg\" (UniqueName: \"kubernetes.io/projected/3c0c22b1-8555-4ff4-9ead-afe3b4a58817-kube-api-access-sxqsg\") pod \"kube-state-metrics-6bd9664c4-9d8mh\" (UID: \"3c0c22b1-8555-4ff4-9ead-afe3b4a58817\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-9d8mh" Aug 04 10:29:01.589592 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.589339 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fda4dea-ac20-4e18-8861-80a87970ca59-sys\") pod \"node-exporter-w798t\" (UID: \"4fda4dea-ac20-4e18-8861-80a87970ca59\") " pod="openshift-monitoring/node-exporter-w798t" Aug 04 10:29:01.589592 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.589367 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsl57\" (UniqueName: \"kubernetes.io/projected/4fda4dea-ac20-4e18-8861-80a87970ca59-kube-api-access-qsl57\") pod \"node-exporter-w798t\" (UID: \"4fda4dea-ac20-4e18-8861-80a87970ca59\") " pod="openshift-monitoring/node-exporter-w798t" Aug 04 10:29:01.589592 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.589393 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4fda4dea-ac20-4e18-8861-80a87970ca59-node-exporter-textfile\") pod \"node-exporter-w798t\" (UID: \"4fda4dea-ac20-4e18-8861-80a87970ca59\") " pod="openshift-monitoring/node-exporter-w798t" Aug 04 10:29:01.589592 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.589423 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4fda4dea-ac20-4e18-8861-80a87970ca59-node-exporter-tls\") pod \"node-exporter-w798t\" (UID: \"4fda4dea-ac20-4e18-8861-80a87970ca59\") " pod="openshift-monitoring/node-exporter-w798t" Aug 04 10:29:01.589592 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.589460 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c0c22b1-8555-4ff4-9ead-afe3b4a58817-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-9d8mh\" (UID: \"3c0c22b1-8555-4ff4-9ead-afe3b4a58817\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-9d8mh" Aug 04 10:29:01.589592 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.589504 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4fda4dea-ac20-4e18-8861-80a87970ca59-root\") pod \"node-exporter-w798t\" (UID: \"4fda4dea-ac20-4e18-8861-80a87970ca59\") " pod="openshift-monitoring/node-exporter-w798t" Aug 04 10:29:01.589592 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.589544 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3c0c22b1-8555-4ff4-9ead-afe3b4a58817-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-9d8mh\" (UID: \"3c0c22b1-8555-4ff4-9ead-afe3b4a58817\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-9d8mh" Aug 04 10:29:01.589592 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.589574 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3c0c22b1-8555-4ff4-9ead-afe3b4a58817-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-9d8mh\" (UID: \"3c0c22b1-8555-4ff4-9ead-afe3b4a58817\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-9d8mh" Aug 04 10:29:01.589987 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.589599 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4fda4dea-ac20-4e18-8861-80a87970ca59-node-exporter-wtmp\") pod \"node-exporter-w798t\" (UID: \"4fda4dea-ac20-4e18-8861-80a87970ca59\") " pod="openshift-monitoring/node-exporter-w798t" Aug 04 10:29:01.589987 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.589753 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4fda4dea-ac20-4e18-8861-80a87970ca59-node-exporter-wtmp\") pod \"node-exporter-w798t\" (UID: \"4fda4dea-ac20-4e18-8861-80a87970ca59\") " pod="openshift-monitoring/node-exporter-w798t" Aug 04 10:29:01.589987 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.589805 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fda4dea-ac20-4e18-8861-80a87970ca59-sys\") pod \"node-exporter-w798t\" (UID: \"4fda4dea-ac20-4e18-8861-80a87970ca59\") " pod="openshift-monitoring/node-exporter-w798t" Aug 04 10:29:01.590142 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.590119 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4fda4dea-ac20-4e18-8861-80a87970ca59-node-exporter-accelerators-collector-config\") pod \"node-exporter-w798t\" (UID: \"4fda4dea-ac20-4e18-8861-80a87970ca59\") " pod="openshift-monitoring/node-exporter-w798t" Aug 04 10:29:01.590142 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.590129 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c0c22b1-8555-4ff4-9ead-afe3b4a58817-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-9d8mh\" (UID: \"3c0c22b1-8555-4ff4-9ead-afe3b4a58817\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-9d8mh" Aug 04 10:29:01.590253 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.590196 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4fda4dea-ac20-4e18-8861-80a87970ca59-root\") pod \"node-exporter-w798t\" (UID: \"4fda4dea-ac20-4e18-8861-80a87970ca59\") " pod="openshift-monitoring/node-exporter-w798t" Aug 04 10:29:01.590306 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.590267 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4fda4dea-ac20-4e18-8861-80a87970ca59-node-exporter-textfile\") pod \"node-exporter-w798t\" (UID: \"4fda4dea-ac20-4e18-8861-80a87970ca59\") " pod="openshift-monitoring/node-exporter-w798t" Aug 04 10:29:01.590717 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.590655 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4fda4dea-ac20-4e18-8861-80a87970ca59-metrics-client-ca\") pod \"node-exporter-w798t\" (UID: \"4fda4dea-ac20-4e18-8861-80a87970ca59\") " pod="openshift-monitoring/node-exporter-w798t" Aug 04 10:29:01.590836 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.590805 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3c0c22b1-8555-4ff4-9ead-afe3b4a58817-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-9d8mh\" (UID: \"3c0c22b1-8555-4ff4-9ead-afe3b4a58817\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-9d8mh" Aug 04 10:29:01.590907 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.590849 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3c0c22b1-8555-4ff4-9ead-afe3b4a58817-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-9d8mh\" (UID: \"3c0c22b1-8555-4ff4-9ead-afe3b4a58817\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-9d8mh" Aug 04 10:29:01.590961 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:29:01.590925 2535 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Aug 04 10:29:01.591009 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:29:01.590988 2535 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c0c22b1-8555-4ff4-9ead-afe3b4a58817-kube-state-metrics-tls podName:3c0c22b1-8555-4ff4-9ead-afe3b4a58817 nodeName:}" failed. No retries permitted until 2026-08-04 10:29:02.090969242 +0000 UTC m=+80.065051752 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/3c0c22b1-8555-4ff4-9ead-afe3b4a58817-kube-state-metrics-tls") pod "kube-state-metrics-6bd9664c4-9d8mh" (UID: "3c0c22b1-8555-4ff4-9ead-afe3b4a58817") : secret "kube-state-metrics-tls" not found Aug 04 10:29:01.592418 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.592396 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3c0c22b1-8555-4ff4-9ead-afe3b4a58817-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-9d8mh\" (UID: \"3c0c22b1-8555-4ff4-9ead-afe3b4a58817\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-9d8mh" Aug 04 10:29:01.592852 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.592829 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4fda4dea-ac20-4e18-8861-80a87970ca59-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w798t\" (UID: \"4fda4dea-ac20-4e18-8861-80a87970ca59\") " pod="openshift-monitoring/node-exporter-w798t" Aug 04 10:29:01.593438 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.593416 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4fda4dea-ac20-4e18-8861-80a87970ca59-node-exporter-tls\") pod \"node-exporter-w798t\" (UID: \"4fda4dea-ac20-4e18-8861-80a87970ca59\") " pod="openshift-monitoring/node-exporter-w798t" Aug 04 10:29:01.600815 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.600689 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxqsg\" (UniqueName: \"kubernetes.io/projected/3c0c22b1-8555-4ff4-9ead-afe3b4a58817-kube-api-access-sxqsg\") pod \"kube-state-metrics-6bd9664c4-9d8mh\" (UID: \"3c0c22b1-8555-4ff4-9ead-afe3b4a58817\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-9d8mh" Aug 04 10:29:01.602666 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.602625 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsl57\" (UniqueName: \"kubernetes.io/projected/4fda4dea-ac20-4e18-8861-80a87970ca59-kube-api-access-qsl57\") pod \"node-exporter-w798t\" (UID: \"4fda4dea-ac20-4e18-8861-80a87970ca59\") " pod="openshift-monitoring/node-exporter-w798t" Aug 04 10:29:01.677259 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.677189 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:01.720842 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.720812 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-w798t" Aug 04 10:29:01.734295 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:29:01.734271 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fda4dea_ac20_4e18_8861_80a87970ca59.slice/crio-5bf51de98ba76b98b0132cabde89a4309f5d284c6ef245b78da9412e883b1585 WatchSource:0}: Error finding container 5bf51de98ba76b98b0132cabde89a4309f5d284c6ef245b78da9412e883b1585: Status 404 returned error can't find the container with id 5bf51de98ba76b98b0132cabde89a4309f5d284c6ef245b78da9412e883b1585 Aug 04 10:29:01.779911 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.779888 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:01.822131 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.822107 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w798t" event={"ID":"4fda4dea-ac20-4e18-8861-80a87970ca59","Type":"ContainerStarted","Data":"5bf51de98ba76b98b0132cabde89a4309f5d284c6ef245b78da9412e883b1585"} Aug 04 10:29:01.959736 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:01.959651 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:02.093744 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.093698 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c0c22b1-8555-4ff4-9ead-afe3b4a58817-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-9d8mh\" (UID: \"3c0c22b1-8555-4ff4-9ead-afe3b4a58817\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-9d8mh" Aug 04 10:29:02.095970 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.095951 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c0c22b1-8555-4ff4-9ead-afe3b4a58817-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-9d8mh\" (UID: \"3c0c22b1-8555-4ff4-9ead-afe3b4a58817\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-9d8mh" Aug 04 10:29:02.105771 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.105720 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:02.314857 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.314828 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-9d8mh" Aug 04 10:29:02.421073 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.420956 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:02.480860 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.480827 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-9d8mh"] Aug 04 10:29:02.484131 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.484092 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 04 10:29:02.487293 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:29:02.487138 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c0c22b1_8555_4ff4_9ead_afe3b4a58817.slice/crio-7bbff229dd650a11656139bd4a8d9d15f9e7fffef0a5a5eb9e37cf2112c984b3 WatchSource:0}: Error finding container 7bbff229dd650a11656139bd4a8d9d15f9e7fffef0a5a5eb9e37cf2112c984b3: Status 404 returned error can't find the container with id 7bbff229dd650a11656139bd4a8d9d15f9e7fffef0a5a5eb9e37cf2112c984b3 Aug 04 10:29:02.488241 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.488223 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.490695 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.490673 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Aug 04 10:29:02.490829 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.490710 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Aug 04 10:29:02.490829 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.490710 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Aug 04 10:29:02.491152 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.491088 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Aug 04 10:29:02.491152 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.491121 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Aug 04 10:29:02.491575 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.491412 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Aug 04 10:29:02.491857 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.491834 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-xx78f\"" Aug 04 10:29:02.494006 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.492472 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Aug 04 10:29:02.496351 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.495828 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Aug 04 10:29:02.505084 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.505054 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Aug 04 10:29:02.508556 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.508534 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 04 10:29:02.513711 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.513685 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:02.599067 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.598978 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/220ac383-255d-4ddc-b2cf-957118bad51b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.599067 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.599023 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-config-volume\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.599067 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.599055 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.599347 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.599079 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrzwk\" (UniqueName: \"kubernetes.io/projected/220ac383-255d-4ddc-b2cf-957118bad51b-kube-api-access-zrzwk\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.599347 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.599134 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.599347 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.599189 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.599347 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.599237 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/220ac383-255d-4ddc-b2cf-957118bad51b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.599347 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.599307 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.599347 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.599347 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-web-config\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.599721 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.599375 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.599721 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.599423 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/220ac383-255d-4ddc-b2cf-957118bad51b-config-out\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.599721 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.599459 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/220ac383-255d-4ddc-b2cf-957118bad51b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.599721 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.599527 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/220ac383-255d-4ddc-b2cf-957118bad51b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.644098 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.644068 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:02.700397 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.700360 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.700600 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.700416 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-web-config\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.700600 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.700442 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.700600 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.700459 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/220ac383-255d-4ddc-b2cf-957118bad51b-config-out\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.700600 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.700481 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/220ac383-255d-4ddc-b2cf-957118bad51b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.700600 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.700539 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/220ac383-255d-4ddc-b2cf-957118bad51b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.700600 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.700586 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/220ac383-255d-4ddc-b2cf-957118bad51b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.700872 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.700608 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-config-volume\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.700872 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.700625 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.700872 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.700640 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrzwk\" (UniqueName: \"kubernetes.io/projected/220ac383-255d-4ddc-b2cf-957118bad51b-kube-api-access-zrzwk\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.700872 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.700661 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.700872 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.700680 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.700872 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.700705 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/220ac383-255d-4ddc-b2cf-957118bad51b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.702509 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.702023 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/220ac383-255d-4ddc-b2cf-957118bad51b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.702509 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.702186 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/220ac383-255d-4ddc-b2cf-957118bad51b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.702684 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.702651 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/220ac383-255d-4ddc-b2cf-957118bad51b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.704558 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.704045 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.704558 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.704388 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-config-volume\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.704558 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.704516 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.704558 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.704542 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/220ac383-255d-4ddc-b2cf-957118bad51b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.704979 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.704874 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-web-config\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.705103 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.705078 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/220ac383-255d-4ddc-b2cf-957118bad51b-config-out\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.705209 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.705185 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.706031 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.706011 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.706169 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.706147 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.710775 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.710752 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrzwk\" (UniqueName: \"kubernetes.io/projected/220ac383-255d-4ddc-b2cf-957118bad51b-kube-api-access-zrzwk\") pod \"alertmanager-main-0\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.743157 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.743121 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:02.805880 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.805849 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:29:02.827740 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:02.827699 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-9d8mh" event={"ID":"3c0c22b1-8555-4ff4-9ead-afe3b4a58817","Type":"ContainerStarted","Data":"7bbff229dd650a11656139bd4a8d9d15f9e7fffef0a5a5eb9e37cf2112c984b3"} Aug 04 10:29:03.032981 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:03.032957 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 04 10:29:03.068377 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:29:03.068342 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod220ac383_255d_4ddc_b2cf_957118bad51b.slice/crio-68cabc48e49c4a00428c68497582d5e5d74802b27b94369e2916c207b6944b7b WatchSource:0}: Error finding container 68cabc48e49c4a00428c68497582d5e5d74802b27b94369e2916c207b6944b7b: Status 404 returned error can't find the container with id 68cabc48e49c4a00428c68497582d5e5d74802b27b94369e2916c207b6944b7b Aug 04 10:29:03.833739 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:03.833709 2535 generic.go:356] "Generic (PLEG): container finished" podID="4fda4dea-ac20-4e18-8861-80a87970ca59" containerID="be017a5c8b3a1d113927087705740232a4735e082594e25640d3b90740721fe6" exitCode=0 Aug 04 10:29:03.834218 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:03.833787 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w798t" event={"ID":"4fda4dea-ac20-4e18-8861-80a87970ca59","Type":"ContainerDied","Data":"be017a5c8b3a1d113927087705740232a4735e082594e25640d3b90740721fe6"} Aug 04 10:29:03.835410 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:03.835386 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"220ac383-255d-4ddc-b2cf-957118bad51b","Type":"ContainerStarted","Data":"68cabc48e49c4a00428c68497582d5e5d74802b27b94369e2916c207b6944b7b"} Aug 04 10:29:04.376102 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:04.376016 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:04.477694 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:04.477630 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:04.841540 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:04.841504 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w798t" event={"ID":"4fda4dea-ac20-4e18-8861-80a87970ca59","Type":"ContainerStarted","Data":"a03b0f7391274770b9565d7fabf7f7af09cd6452000b6e25a298a82a11d3f16b"} Aug 04 10:29:04.841894 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:04.841607 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w798t" event={"ID":"4fda4dea-ac20-4e18-8861-80a87970ca59","Type":"ContainerStarted","Data":"46a02c4352131d01d363abe22424b41e9ecaa0b82ae24055f8ed7be46cd71a1f"} Aug 04 10:29:04.844407 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:04.844373 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-9d8mh" event={"ID":"3c0c22b1-8555-4ff4-9ead-afe3b4a58817","Type":"ContainerStarted","Data":"f2733c010e17fad66f0587ee88724e08606a7fe07edb8003c38fa6704d901e4b"} Aug 04 10:29:04.844535 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:04.844428 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-9d8mh" event={"ID":"3c0c22b1-8555-4ff4-9ead-afe3b4a58817","Type":"ContainerStarted","Data":"cbb8a618feb721d7c6d7d34d0341622139a49edd933590f4e4dbeca081caad44"} Aug 04 10:29:04.844535 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:04.844460 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-9d8mh" event={"ID":"3c0c22b1-8555-4ff4-9ead-afe3b4a58817","Type":"ContainerStarted","Data":"482055bedf188aaefcf7de5f95a90c8bf5b40c19f7cc4f41b27b40687e282963"} Aug 04 10:29:04.846297 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:04.846229 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"220ac383-255d-4ddc-b2cf-957118bad51b","Type":"ContainerStarted","Data":"26a6abfb59d91968bfd0b5c2df9fcc3f4628701da07fa456652e1dcb0d431d1c"} Aug 04 10:29:04.867598 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:04.867557 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-w798t" podStartSLOduration=2.6439989329999998 podStartE2EDuration="3.867543125s" totalImagesPullingTime="1.162081275s" totalInitContainerRuntime="61.462917ms" isStatefulPod=true podCreationTimestamp="2026-08-04 10:29:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:29:04.866220257 +0000 UTC m=+82.840302784" watchObservedRunningTime="2026-08-04 10:29:04.867543125 +0000 UTC m=+82.841625651" Aug 04 10:29:04.889449 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:04.889394 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-9d8mh" podStartSLOduration=2.397759015 podStartE2EDuration="3.889374736s" totalImagesPullingTime="1.491615721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:29:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:29:04.888744454 +0000 UTC m=+82.862827035" watchObservedRunningTime="2026-08-04 10:29:04.889374736 +0000 UTC m=+82.863457264" Aug 04 10:29:04.909679 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:04.909655 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:05.005449 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:05.005363 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:05.851588 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:05.851548 2535 generic.go:356] "Generic (PLEG): container finished" podID="220ac383-255d-4ddc-b2cf-957118bad51b" containerID="26a6abfb59d91968bfd0b5c2df9fcc3f4628701da07fa456652e1dcb0d431d1c" exitCode=0 Aug 04 10:29:05.852060 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:05.851639 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"220ac383-255d-4ddc-b2cf-957118bad51b","Type":"ContainerDied","Data":"26a6abfb59d91968bfd0b5c2df9fcc3f4628701da07fa456652e1dcb0d431d1c"} Aug 04 10:29:05.868505 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:05.868461 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:05.969143 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:05.969114 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:06.154127 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:06.154027 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:06.282343 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:06.282314 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:06.416890 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:06.416812 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-77c974ffcf-s6xxv"] Aug 04 10:29:06.419866 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:06.419852 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-s6xxv" Aug 04 10:29:06.422714 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:06.422694 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Aug 04 10:29:06.422891 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:06.422875 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Aug 04 10:29:06.422968 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:06.422922 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-zb95x\"" Aug 04 10:29:06.426554 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:06.426266 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:06.436755 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:06.436734 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-s6xxv"] Aug 04 10:29:06.497010 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:06.496982 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:06.535825 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:06.535798 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb9zn\" (UniqueName: \"kubernetes.io/projected/45b2091a-93db-4ee5-b0b7-116c9ae5ef72-kube-api-access-wb9zn\") pod \"downloads-77c974ffcf-s6xxv\" (UID: \"45b2091a-93db-4ee5-b0b7-116c9ae5ef72\") " pod="openshift-console/downloads-77c974ffcf-s6xxv" Aug 04 10:29:06.597380 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:06.597348 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:06.636719 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:06.636690 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb9zn\" (UniqueName: \"kubernetes.io/projected/45b2091a-93db-4ee5-b0b7-116c9ae5ef72-kube-api-access-wb9zn\") pod \"downloads-77c974ffcf-s6xxv\" (UID: \"45b2091a-93db-4ee5-b0b7-116c9ae5ef72\") " pod="openshift-console/downloads-77c974ffcf-s6xxv" Aug 04 10:29:06.649566 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:06.649535 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb9zn\" (UniqueName: \"kubernetes.io/projected/45b2091a-93db-4ee5-b0b7-116c9ae5ef72-kube-api-access-wb9zn\") pod \"downloads-77c974ffcf-s6xxv\" (UID: \"45b2091a-93db-4ee5-b0b7-116c9ae5ef72\") " pod="openshift-console/downloads-77c974ffcf-s6xxv" Aug 04 10:29:06.728656 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:06.728578 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-s6xxv" Aug 04 10:29:06.745872 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:06.745841 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:06.880395 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:06.880368 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-s6xxv"] Aug 04 10:29:06.883068 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:29:06.883030 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45b2091a_93db_4ee5_b0b7_116c9ae5ef72.slice/crio-9628ec1fabdb51124cbefb2df3e46f46899171f020b3383cf08dcae399f78ba2 WatchSource:0}: Error finding container 9628ec1fabdb51124cbefb2df3e46f46899171f020b3383cf08dcae399f78ba2: Status 404 returned error can't find the container with id 9628ec1fabdb51124cbefb2df3e46f46899171f020b3383cf08dcae399f78ba2 Aug 04 10:29:06.890903 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:06.890875 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:07.164347 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.164312 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:07.472355 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.472240 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:07.547600 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.547567 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:07.698615 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.698560 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:07.706553 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.706431 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 10:29:07.711429 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.711119 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.719117 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.717555 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Aug 04 10:29:07.719117 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.717820 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Aug 04 10:29:07.719117 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.718148 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Aug 04 10:29:07.719117 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.718361 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Aug 04 10:29:07.719117 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.718535 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Aug 04 10:29:07.719438 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.719139 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Aug 04 10:29:07.719522 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.719469 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Aug 04 10:29:07.719576 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.719549 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Aug 04 10:29:07.719901 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.719756 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Aug 04 10:29:07.719901 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.719765 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Aug 04 10:29:07.719901 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.719812 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Aug 04 10:29:07.720106 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.720006 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-5vpl2\"" Aug 04 10:29:07.720106 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.720022 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-57s78eprjclpa\"" Aug 04 10:29:07.720254 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.720237 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Aug 04 10:29:07.720378 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.720240 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Aug 04 10:29:07.720638 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.720618 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Aug 04 10:29:07.720638 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.720632 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Aug 04 10:29:07.738179 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.736440 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 10:29:07.833919 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.833879 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:07.848997 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.848847 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/16710393-39e5-40af-a484-0c0a881639fd-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.848997 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.848893 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/16710393-39e5-40af-a484-0c0a881639fd-config-out\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.848997 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.848919 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.848997 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.848952 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.849317 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.849292 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.849413 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.849344 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.849413 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.849381 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/16710393-39e5-40af-a484-0c0a881639fd-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.849534 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.849511 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.849614 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.849547 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2z858\" (UniqueName: \"kubernetes.io/projected/16710393-39e5-40af-a484-0c0a881639fd-kube-api-access-2z858\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.849666 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.849610 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.849666 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.849636 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.849666 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.849662 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.849816 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.849684 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.849816 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.849716 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.849816 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.849763 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-config\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.849816 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.849794 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.850003 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.849825 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.850003 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.849852 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.850003 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.849876 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-web-config\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.850003 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.849902 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.863318 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.863290 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-s6xxv" event={"ID":"45b2091a-93db-4ee5-b0b7-116c9ae5ef72","Type":"ContainerStarted","Data":"9628ec1fabdb51124cbefb2df3e46f46899171f020b3383cf08dcae399f78ba2"} Aug 04 10:29:07.933919 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.933892 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:07.951983 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.951760 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/16710393-39e5-40af-a484-0c0a881639fd-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.951983 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.951838 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.951983 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.951870 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2z858\" (UniqueName: \"kubernetes.io/projected/16710393-39e5-40af-a484-0c0a881639fd-kube-api-access-2z858\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.951983 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.951914 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.951983 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.951941 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.951983 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.951966 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.951983 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.951988 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.952381 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.952018 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.952381 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.952042 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-config\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.952381 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.952065 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.952381 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.952092 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.952381 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.952117 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.952381 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.952139 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-web-config\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.952381 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.952163 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.952381 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.952192 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/16710393-39e5-40af-a484-0c0a881639fd-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.952381 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.952215 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/16710393-39e5-40af-a484-0c0a881639fd-config-out\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.952381 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.952236 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.952381 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.952274 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.952381 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.952297 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.952381 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.952324 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.953036 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.952657 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/16710393-39e5-40af-a484-0c0a881639fd-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.953386 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.953364 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.954309 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.954279 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.954859 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.954827 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.955861 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.955837 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.955960 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.955911 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.956370 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.956342 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.959694 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.959273 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.959796 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.959720 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.960439 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.960365 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.960972 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.960932 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.962630 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.962549 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.962630 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.962592 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2z858\" (UniqueName: \"kubernetes.io/projected/16710393-39e5-40af-a484-0c0a881639fd-kube-api-access-2z858\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.963969 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.963946 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-config\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.965614 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.965592 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.966100 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.966077 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/16710393-39e5-40af-a484-0c0a881639fd-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.966233 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.966212 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.966420 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.966390 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/16710393-39e5-40af-a484-0c0a881639fd-config-out\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.966420 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.966398 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-web-config\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:07.966693 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:07.966657 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:08.025826 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:08.025797 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:08.105635 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:08.105552 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:08.223702 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:08.222165 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 10:29:08.226156 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:29:08.226076 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16710393_39e5_40af_a484_0c0a881639fd.slice/crio-5247578cf3446a8758d590f18e0560f29c129881e6131b9316977600e6402f62 WatchSource:0}: Error finding container 5247578cf3446a8758d590f18e0560f29c129881e6131b9316977600e6402f62: Status 404 returned error can't find the container with id 5247578cf3446a8758d590f18e0560f29c129881e6131b9316977600e6402f62 Aug 04 10:29:08.308436 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:08.308405 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:08.405646 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:08.405611 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:08.873101 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:08.873062 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"220ac383-255d-4ddc-b2cf-957118bad51b","Type":"ContainerStarted","Data":"290a7b8811277dbdabd1204f1d5af7722aa0bf900b609df1718cf322b9d29be5"} Aug 04 10:29:08.873258 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:08.873157 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"220ac383-255d-4ddc-b2cf-957118bad51b","Type":"ContainerStarted","Data":"c822b2f68257b50f1553198dab34916eac0ffe071dad205de7ccf4f9fa0e4e56"} Aug 04 10:29:08.873258 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:08.873199 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"220ac383-255d-4ddc-b2cf-957118bad51b","Type":"ContainerStarted","Data":"278660cbfed9f0d31f77f4d47e28b6a375d60c9952b276e4fe8bcd2f1fc39f13"} Aug 04 10:29:08.873258 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:08.873237 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"220ac383-255d-4ddc-b2cf-957118bad51b","Type":"ContainerStarted","Data":"462302712d667308ace61ff88f69507ba0a495201e61aeef1ceb0ce61e03d377"} Aug 04 10:29:08.873425 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:08.873267 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"220ac383-255d-4ddc-b2cf-957118bad51b","Type":"ContainerStarted","Data":"717b4eaca372e9653bbfed6491f9ee429d01caf624f2160583a6972c4224e5e0"} Aug 04 10:29:08.874752 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:08.874722 2535 generic.go:356] "Generic (PLEG): container finished" podID="16710393-39e5-40af-a484-0c0a881639fd" containerID="f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63" exitCode=0 Aug 04 10:29:08.874876 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:08.874764 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16710393-39e5-40af-a484-0c0a881639fd","Type":"ContainerDied","Data":"f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63"} Aug 04 10:29:08.874876 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:08.874830 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16710393-39e5-40af-a484-0c0a881639fd","Type":"ContainerStarted","Data":"5247578cf3446a8758d590f18e0560f29c129881e6131b9316977600e6402f62"} Aug 04 10:29:09.133683 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:09.133598 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:09.234674 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:09.234634 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:09.885724 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:09.885519 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"220ac383-255d-4ddc-b2cf-957118bad51b","Type":"ContainerStarted","Data":"ba5d389f5e284e19c081f25d3cf0cd81a2f3c3e63f49774fde1026371208b8c2"} Aug 04 10:29:09.920695 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:09.920550 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.467979499 podStartE2EDuration="7.920530688s" totalImagesPullingTime="4.43340625s" totalInitContainerRuntime="19.144939ms" isStatefulPod=false podCreationTimestamp="2026-08-04 10:29:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:29:09.919376207 +0000 UTC m=+87.893458759" watchObservedRunningTime="2026-08-04 10:29:09.920530688 +0000 UTC m=+87.894613261" Aug 04 10:29:10.768889 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:10.768849 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-74d45d67dc-rb8sl" Aug 04 10:29:10.962110 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:10.962045 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:11.055811 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:11.055774 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:11.117417 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:11.117350 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:12.904294 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:12.904259 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16710393-39e5-40af-a484-0c0a881639fd","Type":"ContainerStarted","Data":"fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884"} Aug 04 10:29:13.911935 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:13.911844 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16710393-39e5-40af-a484-0c0a881639fd","Type":"ContainerStarted","Data":"fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba"} Aug 04 10:29:15.924904 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:15.924863 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16710393-39e5-40af-a484-0c0a881639fd","Type":"ContainerStarted","Data":"34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe"} Aug 04 10:29:15.925336 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:15.924947 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16710393-39e5-40af-a484-0c0a881639fd","Type":"ContainerStarted","Data":"c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4"} Aug 04 10:29:15.925336 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:15.924987 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16710393-39e5-40af-a484-0c0a881639fd","Type":"ContainerStarted","Data":"cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b"} Aug 04 10:29:15.925336 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:15.925023 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16710393-39e5-40af-a484-0c0a881639fd","Type":"ContainerStarted","Data":"9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770"} Aug 04 10:29:15.962916 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:15.962850 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.900816307 podStartE2EDuration="8.962833594s" totalImagesPullingTime="6.035120043s" totalInitContainerRuntime="26.897244ms" isStatefulPod=false podCreationTimestamp="2026-08-04 10:29:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:29:15.960714005 +0000 UTC m=+93.934796557" watchObservedRunningTime="2026-08-04 10:29:15.962833594 +0000 UTC m=+93.936916115" Aug 04 10:29:16.612841 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:16.612796 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:16.671901 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:16.671863 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:18.026611 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:18.026579 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:29:18.165754 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:18.165721 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:18.228354 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:18.228294 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:19.582695 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:19.582657 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:19.641146 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:19.641116 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:20.615402 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:20.615365 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:20.682376 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:20.682341 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:22.116953 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:22.116919 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:22.218128 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:22.218089 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:24.971925 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:24.971874 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-s6xxv" event={"ID":"45b2091a-93db-4ee5-b0b7-116c9ae5ef72","Type":"ContainerStarted","Data":"ec2529bae61e8925ca251d603ac24c93c089672a5505d5f508e189c1a35ee569"} Aug 04 10:29:24.972394 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:24.972093 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-77c974ffcf-s6xxv" Aug 04 10:29:24.979884 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:24.979863 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-77c974ffcf-s6xxv" Aug 04 10:29:24.991527 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:24.990968 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-77c974ffcf-s6xxv" podStartSLOduration=1.126348046 podStartE2EDuration="18.990943387s" totalImagesPullingTime="17.864595341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:29:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:29:24.990916646 +0000 UTC m=+102.964999175" watchObservedRunningTime="2026-08-04 10:29:24.990943387 +0000 UTC m=+102.965025915" Aug 04 10:29:25.608250 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:25.608185 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:25.737962 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:25.737927 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:25.924791 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:25.924700 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:25.983649 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:25.983615 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:27.724209 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:27.724171 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:27.801408 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:27.801379 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2rc2x" Aug 04 10:29:27.822767 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:27.822736 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:31.353431 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:31.353397 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:31.461305 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:31.461270 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:31.519343 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:31.519307 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:31.752647 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:31.752568 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:31.851342 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:31.851307 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:31.914540 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:31.914502 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:35.623811 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:35.623775 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:35.686693 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:35.686660 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:37.210967 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:37.210938 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ttzdq_1955136f-4af9-4ab3-81df-790fa97cddab/serve-healthcheck-canary/0.log" Aug 04 10:29:42.197384 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:42.197340 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:42.302102 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:42.302068 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:42.385995 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:42.385967 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:50.612307 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:50.612252 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:29:50.680189 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:29:50.680157 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:30:08.026230 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:08.026192 2535 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:08.092782 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:08.092754 2535 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:08.167397 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:08.167372 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:17.547737 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:17.547703 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:30:17.604276 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:17.604243 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:30:21.715742 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:21.715703 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:30:21.861120 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:21.861085 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 04 10:30:21.861471 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:21.861442 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="220ac383-255d-4ddc-b2cf-957118bad51b" containerName="alertmanager" containerID="cri-o://717b4eaca372e9653bbfed6491f9ee429d01caf624f2160583a6972c4224e5e0" gracePeriod=120 Aug 04 10:30:21.861580 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:21.861510 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="220ac383-255d-4ddc-b2cf-957118bad51b" containerName="kube-rbac-proxy-metric" containerID="cri-o://290a7b8811277dbdabd1204f1d5af7722aa0bf900b609df1718cf322b9d29be5" gracePeriod=120 Aug 04 10:30:21.861580 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:21.861528 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="220ac383-255d-4ddc-b2cf-957118bad51b" containerName="kube-rbac-proxy-web" containerID="cri-o://278660cbfed9f0d31f77f4d47e28b6a375d60c9952b276e4fe8bcd2f1fc39f13" gracePeriod=120 Aug 04 10:30:21.861580 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:21.861542 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="220ac383-255d-4ddc-b2cf-957118bad51b" containerName="config-reloader" containerID="cri-o://462302712d667308ace61ff88f69507ba0a495201e61aeef1ceb0ce61e03d377" gracePeriod=120 Aug 04 10:30:21.861580 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:21.861529 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="220ac383-255d-4ddc-b2cf-957118bad51b" containerName="kube-rbac-proxy" containerID="cri-o://c822b2f68257b50f1553198dab34916eac0ffe071dad205de7ccf4f9fa0e4e56" gracePeriod=120 Aug 04 10:30:21.861760 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:21.861636 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="220ac383-255d-4ddc-b2cf-957118bad51b" containerName="prom-label-proxy" containerID="cri-o://ba5d389f5e284e19c081f25d3cf0cd81a2f3c3e63f49774fde1026371208b8c2" gracePeriod=120 Aug 04 10:30:22.207732 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:22.207696 2535 generic.go:356] "Generic (PLEG): container finished" podID="220ac383-255d-4ddc-b2cf-957118bad51b" containerID="ba5d389f5e284e19c081f25d3cf0cd81a2f3c3e63f49774fde1026371208b8c2" exitCode=0 Aug 04 10:30:22.207732 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:22.207724 2535 generic.go:356] "Generic (PLEG): container finished" podID="220ac383-255d-4ddc-b2cf-957118bad51b" containerID="c822b2f68257b50f1553198dab34916eac0ffe071dad205de7ccf4f9fa0e4e56" exitCode=0 Aug 04 10:30:22.207732 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:22.207733 2535 generic.go:356] "Generic (PLEG): container finished" podID="220ac383-255d-4ddc-b2cf-957118bad51b" containerID="462302712d667308ace61ff88f69507ba0a495201e61aeef1ceb0ce61e03d377" exitCode=0 Aug 04 10:30:22.207732 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:22.207740 2535 generic.go:356] "Generic (PLEG): container finished" podID="220ac383-255d-4ddc-b2cf-957118bad51b" containerID="717b4eaca372e9653bbfed6491f9ee429d01caf624f2160583a6972c4224e5e0" exitCode=0 Aug 04 10:30:22.207990 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:22.207765 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"220ac383-255d-4ddc-b2cf-957118bad51b","Type":"ContainerDied","Data":"ba5d389f5e284e19c081f25d3cf0cd81a2f3c3e63f49774fde1026371208b8c2"} Aug 04 10:30:22.207990 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:22.207828 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"220ac383-255d-4ddc-b2cf-957118bad51b","Type":"ContainerDied","Data":"c822b2f68257b50f1553198dab34916eac0ffe071dad205de7ccf4f9fa0e4e56"} Aug 04 10:30:22.207990 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:22.207855 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"220ac383-255d-4ddc-b2cf-957118bad51b","Type":"ContainerDied","Data":"462302712d667308ace61ff88f69507ba0a495201e61aeef1ceb0ce61e03d377"} Aug 04 10:30:22.207990 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:22.207880 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"220ac383-255d-4ddc-b2cf-957118bad51b","Type":"ContainerDied","Data":"717b4eaca372e9653bbfed6491f9ee429d01caf624f2160583a6972c4224e5e0"} Aug 04 10:30:23.096892 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.096868 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:30:23.209445 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.209367 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-secret-alertmanager-kube-rbac-proxy-web\") pod \"220ac383-255d-4ddc-b2cf-957118bad51b\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " Aug 04 10:30:23.209445 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.209426 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-secret-alertmanager-main-tls\") pod \"220ac383-255d-4ddc-b2cf-957118bad51b\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " Aug 04 10:30:23.209445 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.209447 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-config-volume\") pod \"220ac383-255d-4ddc-b2cf-957118bad51b\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " Aug 04 10:30:23.209738 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.209474 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/220ac383-255d-4ddc-b2cf-957118bad51b-config-out\") pod \"220ac383-255d-4ddc-b2cf-957118bad51b\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " Aug 04 10:30:23.209738 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.209523 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/220ac383-255d-4ddc-b2cf-957118bad51b-metrics-client-ca\") pod \"220ac383-255d-4ddc-b2cf-957118bad51b\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " Aug 04 10:30:23.209738 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.209547 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/220ac383-255d-4ddc-b2cf-957118bad51b-alertmanager-main-db\") pod \"220ac383-255d-4ddc-b2cf-957118bad51b\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " Aug 04 10:30:23.209738 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.209573 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/220ac383-255d-4ddc-b2cf-957118bad51b-alertmanager-trusted-ca-bundle\") pod \"220ac383-255d-4ddc-b2cf-957118bad51b\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " Aug 04 10:30:23.209738 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.209613 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/220ac383-255d-4ddc-b2cf-957118bad51b-tls-assets\") pod \"220ac383-255d-4ddc-b2cf-957118bad51b\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " Aug 04 10:30:23.209738 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.209636 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-web-config\") pod \"220ac383-255d-4ddc-b2cf-957118bad51b\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " Aug 04 10:30:23.209738 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.209662 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-cluster-tls-config\") pod \"220ac383-255d-4ddc-b2cf-957118bad51b\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " Aug 04 10:30:23.209738 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.209697 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"220ac383-255d-4ddc-b2cf-957118bad51b\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " Aug 04 10:30:23.210133 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.209758 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrzwk\" (UniqueName: \"kubernetes.io/projected/220ac383-255d-4ddc-b2cf-957118bad51b-kube-api-access-zrzwk\") pod \"220ac383-255d-4ddc-b2cf-957118bad51b\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " Aug 04 10:30:23.210133 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.209803 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-secret-alertmanager-kube-rbac-proxy\") pod \"220ac383-255d-4ddc-b2cf-957118bad51b\" (UID: \"220ac383-255d-4ddc-b2cf-957118bad51b\") " Aug 04 10:30:23.210133 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.209990 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/220ac383-255d-4ddc-b2cf-957118bad51b-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "220ac383-255d-4ddc-b2cf-957118bad51b" (UID: "220ac383-255d-4ddc-b2cf-957118bad51b"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:30:23.211430 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.211201 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/220ac383-255d-4ddc-b2cf-957118bad51b-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "220ac383-255d-4ddc-b2cf-957118bad51b" (UID: "220ac383-255d-4ddc-b2cf-957118bad51b"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:30:23.211586 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.211429 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/220ac383-255d-4ddc-b2cf-957118bad51b-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "220ac383-255d-4ddc-b2cf-957118bad51b" (UID: "220ac383-255d-4ddc-b2cf-957118bad51b"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:30:23.213195 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.213151 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "220ac383-255d-4ddc-b2cf-957118bad51b" (UID: "220ac383-255d-4ddc-b2cf-957118bad51b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 10:30:23.213584 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.213554 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/220ac383-255d-4ddc-b2cf-957118bad51b-config-out" (OuterVolumeSpecName: "config-out") pod "220ac383-255d-4ddc-b2cf-957118bad51b" (UID: "220ac383-255d-4ddc-b2cf-957118bad51b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:30:23.213697 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.213626 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-config-volume" (OuterVolumeSpecName: "config-volume") pod "220ac383-255d-4ddc-b2cf-957118bad51b" (UID: "220ac383-255d-4ddc-b2cf-957118bad51b"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 10:30:23.213697 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.213644 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "220ac383-255d-4ddc-b2cf-957118bad51b" (UID: "220ac383-255d-4ddc-b2cf-957118bad51b"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 10:30:23.213818 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.213742 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "220ac383-255d-4ddc-b2cf-957118bad51b" (UID: "220ac383-255d-4ddc-b2cf-957118bad51b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 10:30:23.214306 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.214280 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/220ac383-255d-4ddc-b2cf-957118bad51b-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "220ac383-255d-4ddc-b2cf-957118bad51b" (UID: "220ac383-255d-4ddc-b2cf-957118bad51b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:30:23.214420 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.214335 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/220ac383-255d-4ddc-b2cf-957118bad51b-kube-api-access-zrzwk" (OuterVolumeSpecName: "kube-api-access-zrzwk") pod "220ac383-255d-4ddc-b2cf-957118bad51b" (UID: "220ac383-255d-4ddc-b2cf-957118bad51b"). InnerVolumeSpecName "kube-api-access-zrzwk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:30:23.215271 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.215241 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "220ac383-255d-4ddc-b2cf-957118bad51b" (UID: "220ac383-255d-4ddc-b2cf-957118bad51b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 10:30:23.217105 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.217076 2535 generic.go:356] "Generic (PLEG): container finished" podID="220ac383-255d-4ddc-b2cf-957118bad51b" containerID="290a7b8811277dbdabd1204f1d5af7722aa0bf900b609df1718cf322b9d29be5" exitCode=0 Aug 04 10:30:23.217105 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.217104 2535 generic.go:356] "Generic (PLEG): container finished" podID="220ac383-255d-4ddc-b2cf-957118bad51b" containerID="278660cbfed9f0d31f77f4d47e28b6a375d60c9952b276e4fe8bcd2f1fc39f13" exitCode=0 Aug 04 10:30:23.217268 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.217154 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"220ac383-255d-4ddc-b2cf-957118bad51b","Type":"ContainerDied","Data":"290a7b8811277dbdabd1204f1d5af7722aa0bf900b609df1718cf322b9d29be5"} Aug 04 10:30:23.217268 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.217166 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Aug 04 10:30:23.217381 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.217273 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"220ac383-255d-4ddc-b2cf-957118bad51b","Type":"ContainerDied","Data":"278660cbfed9f0d31f77f4d47e28b6a375d60c9952b276e4fe8bcd2f1fc39f13"} Aug 04 10:30:23.217381 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.217318 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"220ac383-255d-4ddc-b2cf-957118bad51b","Type":"ContainerDied","Data":"68cabc48e49c4a00428c68497582d5e5d74802b27b94369e2916c207b6944b7b"} Aug 04 10:30:23.217381 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.217348 2535 scope.go:117] "RemoveContainer" containerID="ba5d389f5e284e19c081f25d3cf0cd81a2f3c3e63f49774fde1026371208b8c2" Aug 04 10:30:23.217877 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.217849 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "220ac383-255d-4ddc-b2cf-957118bad51b" (UID: "220ac383-255d-4ddc-b2cf-957118bad51b"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 10:30:23.225758 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.225736 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-web-config" (OuterVolumeSpecName: "web-config") pod "220ac383-255d-4ddc-b2cf-957118bad51b" (UID: "220ac383-255d-4ddc-b2cf-957118bad51b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 10:30:23.226673 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.226660 2535 scope.go:117] "RemoveContainer" containerID="290a7b8811277dbdabd1204f1d5af7722aa0bf900b609df1718cf322b9d29be5" Aug 04 10:30:23.233505 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.233474 2535 scope.go:117] "RemoveContainer" containerID="c822b2f68257b50f1553198dab34916eac0ffe071dad205de7ccf4f9fa0e4e56" Aug 04 10:30:23.239708 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.239692 2535 scope.go:117] "RemoveContainer" containerID="278660cbfed9f0d31f77f4d47e28b6a375d60c9952b276e4fe8bcd2f1fc39f13" Aug 04 10:30:23.245737 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.245707 2535 scope.go:117] "RemoveContainer" containerID="462302712d667308ace61ff88f69507ba0a495201e61aeef1ceb0ce61e03d377" Aug 04 10:30:23.252169 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.252153 2535 scope.go:117] "RemoveContainer" containerID="717b4eaca372e9653bbfed6491f9ee429d01caf624f2160583a6972c4224e5e0" Aug 04 10:30:23.258710 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.258695 2535 scope.go:117] "RemoveContainer" containerID="26a6abfb59d91968bfd0b5c2df9fcc3f4628701da07fa456652e1dcb0d431d1c" Aug 04 10:30:23.265073 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.265059 2535 scope.go:117] "RemoveContainer" containerID="ba5d389f5e284e19c081f25d3cf0cd81a2f3c3e63f49774fde1026371208b8c2" Aug 04 10:30:23.265307 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:30:23.265289 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba5d389f5e284e19c081f25d3cf0cd81a2f3c3e63f49774fde1026371208b8c2\": container with ID starting with ba5d389f5e284e19c081f25d3cf0cd81a2f3c3e63f49774fde1026371208b8c2 not found: ID does not exist" containerID="ba5d389f5e284e19c081f25d3cf0cd81a2f3c3e63f49774fde1026371208b8c2" Aug 04 10:30:23.265349 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.265314 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba5d389f5e284e19c081f25d3cf0cd81a2f3c3e63f49774fde1026371208b8c2"} err="failed to get container status \"ba5d389f5e284e19c081f25d3cf0cd81a2f3c3e63f49774fde1026371208b8c2\": rpc error: code = NotFound desc = could not find container \"ba5d389f5e284e19c081f25d3cf0cd81a2f3c3e63f49774fde1026371208b8c2\": container with ID starting with ba5d389f5e284e19c081f25d3cf0cd81a2f3c3e63f49774fde1026371208b8c2 not found: ID does not exist" Aug 04 10:30:23.265349 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.265345 2535 scope.go:117] "RemoveContainer" containerID="290a7b8811277dbdabd1204f1d5af7722aa0bf900b609df1718cf322b9d29be5" Aug 04 10:30:23.265565 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:30:23.265549 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"290a7b8811277dbdabd1204f1d5af7722aa0bf900b609df1718cf322b9d29be5\": container with ID starting with 290a7b8811277dbdabd1204f1d5af7722aa0bf900b609df1718cf322b9d29be5 not found: ID does not exist" containerID="290a7b8811277dbdabd1204f1d5af7722aa0bf900b609df1718cf322b9d29be5" Aug 04 10:30:23.265613 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.265569 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"290a7b8811277dbdabd1204f1d5af7722aa0bf900b609df1718cf322b9d29be5"} err="failed to get container status \"290a7b8811277dbdabd1204f1d5af7722aa0bf900b609df1718cf322b9d29be5\": rpc error: code = NotFound desc = could not find container \"290a7b8811277dbdabd1204f1d5af7722aa0bf900b609df1718cf322b9d29be5\": container with ID starting with 290a7b8811277dbdabd1204f1d5af7722aa0bf900b609df1718cf322b9d29be5 not found: ID does not exist" Aug 04 10:30:23.265613 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.265584 2535 scope.go:117] "RemoveContainer" containerID="c822b2f68257b50f1553198dab34916eac0ffe071dad205de7ccf4f9fa0e4e56" Aug 04 10:30:23.265807 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:30:23.265789 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c822b2f68257b50f1553198dab34916eac0ffe071dad205de7ccf4f9fa0e4e56\": container with ID starting with c822b2f68257b50f1553198dab34916eac0ffe071dad205de7ccf4f9fa0e4e56 not found: ID does not exist" containerID="c822b2f68257b50f1553198dab34916eac0ffe071dad205de7ccf4f9fa0e4e56" Aug 04 10:30:23.265866 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.265815 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c822b2f68257b50f1553198dab34916eac0ffe071dad205de7ccf4f9fa0e4e56"} err="failed to get container status \"c822b2f68257b50f1553198dab34916eac0ffe071dad205de7ccf4f9fa0e4e56\": rpc error: code = NotFound desc = could not find container \"c822b2f68257b50f1553198dab34916eac0ffe071dad205de7ccf4f9fa0e4e56\": container with ID starting with c822b2f68257b50f1553198dab34916eac0ffe071dad205de7ccf4f9fa0e4e56 not found: ID does not exist" Aug 04 10:30:23.265866 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.265836 2535 scope.go:117] "RemoveContainer" containerID="278660cbfed9f0d31f77f4d47e28b6a375d60c9952b276e4fe8bcd2f1fc39f13" Aug 04 10:30:23.266056 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:30:23.266040 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"278660cbfed9f0d31f77f4d47e28b6a375d60c9952b276e4fe8bcd2f1fc39f13\": container with ID starting with 278660cbfed9f0d31f77f4d47e28b6a375d60c9952b276e4fe8bcd2f1fc39f13 not found: ID does not exist" containerID="278660cbfed9f0d31f77f4d47e28b6a375d60c9952b276e4fe8bcd2f1fc39f13" Aug 04 10:30:23.266092 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.266059 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"278660cbfed9f0d31f77f4d47e28b6a375d60c9952b276e4fe8bcd2f1fc39f13"} err="failed to get container status \"278660cbfed9f0d31f77f4d47e28b6a375d60c9952b276e4fe8bcd2f1fc39f13\": rpc error: code = NotFound desc = could not find container \"278660cbfed9f0d31f77f4d47e28b6a375d60c9952b276e4fe8bcd2f1fc39f13\": container with ID starting with 278660cbfed9f0d31f77f4d47e28b6a375d60c9952b276e4fe8bcd2f1fc39f13 not found: ID does not exist" Aug 04 10:30:23.266092 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.266072 2535 scope.go:117] "RemoveContainer" containerID="462302712d667308ace61ff88f69507ba0a495201e61aeef1ceb0ce61e03d377" Aug 04 10:30:23.266274 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:30:23.266254 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"462302712d667308ace61ff88f69507ba0a495201e61aeef1ceb0ce61e03d377\": container with ID starting with 462302712d667308ace61ff88f69507ba0a495201e61aeef1ceb0ce61e03d377 not found: ID does not exist" containerID="462302712d667308ace61ff88f69507ba0a495201e61aeef1ceb0ce61e03d377" Aug 04 10:30:23.266339 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.266281 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"462302712d667308ace61ff88f69507ba0a495201e61aeef1ceb0ce61e03d377"} err="failed to get container status \"462302712d667308ace61ff88f69507ba0a495201e61aeef1ceb0ce61e03d377\": rpc error: code = NotFound desc = could not find container \"462302712d667308ace61ff88f69507ba0a495201e61aeef1ceb0ce61e03d377\": container with ID starting with 462302712d667308ace61ff88f69507ba0a495201e61aeef1ceb0ce61e03d377 not found: ID does not exist" Aug 04 10:30:23.266339 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.266300 2535 scope.go:117] "RemoveContainer" containerID="717b4eaca372e9653bbfed6491f9ee429d01caf624f2160583a6972c4224e5e0" Aug 04 10:30:23.266613 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:30:23.266596 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"717b4eaca372e9653bbfed6491f9ee429d01caf624f2160583a6972c4224e5e0\": container with ID starting with 717b4eaca372e9653bbfed6491f9ee429d01caf624f2160583a6972c4224e5e0 not found: ID does not exist" containerID="717b4eaca372e9653bbfed6491f9ee429d01caf624f2160583a6972c4224e5e0" Aug 04 10:30:23.266663 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.266616 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"717b4eaca372e9653bbfed6491f9ee429d01caf624f2160583a6972c4224e5e0"} err="failed to get container status \"717b4eaca372e9653bbfed6491f9ee429d01caf624f2160583a6972c4224e5e0\": rpc error: code = NotFound desc = could not find container \"717b4eaca372e9653bbfed6491f9ee429d01caf624f2160583a6972c4224e5e0\": container with ID starting with 717b4eaca372e9653bbfed6491f9ee429d01caf624f2160583a6972c4224e5e0 not found: ID does not exist" Aug 04 10:30:23.266663 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.266631 2535 scope.go:117] "RemoveContainer" containerID="26a6abfb59d91968bfd0b5c2df9fcc3f4628701da07fa456652e1dcb0d431d1c" Aug 04 10:30:23.266846 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:30:23.266826 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26a6abfb59d91968bfd0b5c2df9fcc3f4628701da07fa456652e1dcb0d431d1c\": container with ID starting with 26a6abfb59d91968bfd0b5c2df9fcc3f4628701da07fa456652e1dcb0d431d1c not found: ID does not exist" containerID="26a6abfb59d91968bfd0b5c2df9fcc3f4628701da07fa456652e1dcb0d431d1c" Aug 04 10:30:23.266929 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.266848 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26a6abfb59d91968bfd0b5c2df9fcc3f4628701da07fa456652e1dcb0d431d1c"} err="failed to get container status \"26a6abfb59d91968bfd0b5c2df9fcc3f4628701da07fa456652e1dcb0d431d1c\": rpc error: code = NotFound desc = could not find container \"26a6abfb59d91968bfd0b5c2df9fcc3f4628701da07fa456652e1dcb0d431d1c\": container with ID starting with 26a6abfb59d91968bfd0b5c2df9fcc3f4628701da07fa456652e1dcb0d431d1c not found: ID does not exist" Aug 04 10:30:23.266929 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.266861 2535 scope.go:117] "RemoveContainer" containerID="ba5d389f5e284e19c081f25d3cf0cd81a2f3c3e63f49774fde1026371208b8c2" Aug 04 10:30:23.267031 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.267013 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba5d389f5e284e19c081f25d3cf0cd81a2f3c3e63f49774fde1026371208b8c2"} err="failed to get container status \"ba5d389f5e284e19c081f25d3cf0cd81a2f3c3e63f49774fde1026371208b8c2\": rpc error: code = NotFound desc = could not find container \"ba5d389f5e284e19c081f25d3cf0cd81a2f3c3e63f49774fde1026371208b8c2\": container with ID starting with ba5d389f5e284e19c081f25d3cf0cd81a2f3c3e63f49774fde1026371208b8c2 not found: ID does not exist" Aug 04 10:30:23.267070 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.267032 2535 scope.go:117] "RemoveContainer" containerID="290a7b8811277dbdabd1204f1d5af7722aa0bf900b609df1718cf322b9d29be5" Aug 04 10:30:23.267172 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.267158 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"290a7b8811277dbdabd1204f1d5af7722aa0bf900b609df1718cf322b9d29be5"} err="failed to get container status \"290a7b8811277dbdabd1204f1d5af7722aa0bf900b609df1718cf322b9d29be5\": rpc error: code = NotFound desc = could not find container \"290a7b8811277dbdabd1204f1d5af7722aa0bf900b609df1718cf322b9d29be5\": container with ID starting with 290a7b8811277dbdabd1204f1d5af7722aa0bf900b609df1718cf322b9d29be5 not found: ID does not exist" Aug 04 10:30:23.267172 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.267172 2535 scope.go:117] "RemoveContainer" containerID="c822b2f68257b50f1553198dab34916eac0ffe071dad205de7ccf4f9fa0e4e56" Aug 04 10:30:23.267368 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.267350 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c822b2f68257b50f1553198dab34916eac0ffe071dad205de7ccf4f9fa0e4e56"} err="failed to get container status \"c822b2f68257b50f1553198dab34916eac0ffe071dad205de7ccf4f9fa0e4e56\": rpc error: code = NotFound desc = could not find container \"c822b2f68257b50f1553198dab34916eac0ffe071dad205de7ccf4f9fa0e4e56\": container with ID starting with c822b2f68257b50f1553198dab34916eac0ffe071dad205de7ccf4f9fa0e4e56 not found: ID does not exist" Aug 04 10:30:23.267417 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.267369 2535 scope.go:117] "RemoveContainer" containerID="278660cbfed9f0d31f77f4d47e28b6a375d60c9952b276e4fe8bcd2f1fc39f13" Aug 04 10:30:23.267640 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.267621 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"278660cbfed9f0d31f77f4d47e28b6a375d60c9952b276e4fe8bcd2f1fc39f13"} err="failed to get container status \"278660cbfed9f0d31f77f4d47e28b6a375d60c9952b276e4fe8bcd2f1fc39f13\": rpc error: code = NotFound desc = could not find container \"278660cbfed9f0d31f77f4d47e28b6a375d60c9952b276e4fe8bcd2f1fc39f13\": container with ID starting with 278660cbfed9f0d31f77f4d47e28b6a375d60c9952b276e4fe8bcd2f1fc39f13 not found: ID does not exist" Aug 04 10:30:23.267690 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.267640 2535 scope.go:117] "RemoveContainer" containerID="462302712d667308ace61ff88f69507ba0a495201e61aeef1ceb0ce61e03d377" Aug 04 10:30:23.267875 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.267856 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"462302712d667308ace61ff88f69507ba0a495201e61aeef1ceb0ce61e03d377"} err="failed to get container status \"462302712d667308ace61ff88f69507ba0a495201e61aeef1ceb0ce61e03d377\": rpc error: code = NotFound desc = could not find container \"462302712d667308ace61ff88f69507ba0a495201e61aeef1ceb0ce61e03d377\": container with ID starting with 462302712d667308ace61ff88f69507ba0a495201e61aeef1ceb0ce61e03d377 not found: ID does not exist" Aug 04 10:30:23.267942 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.267888 2535 scope.go:117] "RemoveContainer" containerID="717b4eaca372e9653bbfed6491f9ee429d01caf624f2160583a6972c4224e5e0" Aug 04 10:30:23.268133 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.268113 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"717b4eaca372e9653bbfed6491f9ee429d01caf624f2160583a6972c4224e5e0"} err="failed to get container status \"717b4eaca372e9653bbfed6491f9ee429d01caf624f2160583a6972c4224e5e0\": rpc error: code = NotFound desc = could not find container \"717b4eaca372e9653bbfed6491f9ee429d01caf624f2160583a6972c4224e5e0\": container with ID starting with 717b4eaca372e9653bbfed6491f9ee429d01caf624f2160583a6972c4224e5e0 not found: ID does not exist" Aug 04 10:30:23.268181 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.268132 2535 scope.go:117] "RemoveContainer" containerID="26a6abfb59d91968bfd0b5c2df9fcc3f4628701da07fa456652e1dcb0d431d1c" Aug 04 10:30:23.268316 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.268301 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26a6abfb59d91968bfd0b5c2df9fcc3f4628701da07fa456652e1dcb0d431d1c"} err="failed to get container status \"26a6abfb59d91968bfd0b5c2df9fcc3f4628701da07fa456652e1dcb0d431d1c\": rpc error: code = NotFound desc = could not find container \"26a6abfb59d91968bfd0b5c2df9fcc3f4628701da07fa456652e1dcb0d431d1c\": container with ID starting with 26a6abfb59d91968bfd0b5c2df9fcc3f4628701da07fa456652e1dcb0d431d1c not found: ID does not exist" Aug 04 10:30:23.310659 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.310633 2535 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-cluster-tls-config\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:23.310659 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.310655 2535 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:23.310787 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.310665 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrzwk\" (UniqueName: \"kubernetes.io/projected/220ac383-255d-4ddc-b2cf-957118bad51b-kube-api-access-zrzwk\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:23.310787 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.310674 2535 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:23.310787 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.310683 2535 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:23.310787 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.310692 2535 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-secret-alertmanager-main-tls\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:23.310787 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.310701 2535 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:23.310787 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.310708 2535 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/220ac383-255d-4ddc-b2cf-957118bad51b-config-out\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:23.310787 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.310716 2535 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/220ac383-255d-4ddc-b2cf-957118bad51b-metrics-client-ca\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:23.310787 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.310724 2535 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/220ac383-255d-4ddc-b2cf-957118bad51b-alertmanager-main-db\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:23.310787 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.310733 2535 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/220ac383-255d-4ddc-b2cf-957118bad51b-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:23.310787 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.310740 2535 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/220ac383-255d-4ddc-b2cf-957118bad51b-tls-assets\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:23.310787 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.310748 2535 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/220ac383-255d-4ddc-b2cf-957118bad51b-web-config\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:23.542942 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.542908 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 04 10:30:23.547266 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:23.547243 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 04 10:30:24.479024 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:24.478994 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="220ac383-255d-4ddc-b2cf-957118bad51b" path="/var/lib/kubelet/pods/220ac383-255d-4ddc-b2cf-957118bad51b/volumes" Aug 04 10:30:26.116190 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.116156 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 10:30:26.117088 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.117032 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="16710393-39e5-40af-a484-0c0a881639fd" containerName="prometheus" containerID="cri-o://fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884" gracePeriod=600 Aug 04 10:30:26.117469 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.117436 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="16710393-39e5-40af-a484-0c0a881639fd" containerName="config-reloader" containerID="cri-o://fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba" gracePeriod=600 Aug 04 10:30:26.117617 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.117373 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="16710393-39e5-40af-a484-0c0a881639fd" containerName="kube-rbac-proxy-thanos" containerID="cri-o://34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe" gracePeriod=600 Aug 04 10:30:26.117675 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.117394 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="16710393-39e5-40af-a484-0c0a881639fd" containerName="thanos-sidecar" containerID="cri-o://9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770" gracePeriod=600 Aug 04 10:30:26.117731 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.117409 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="16710393-39e5-40af-a484-0c0a881639fd" containerName="kube-rbac-proxy-web" containerID="cri-o://cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b" gracePeriod=600 Aug 04 10:30:26.117835 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.117228 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="16710393-39e5-40af-a484-0c0a881639fd" containerName="kube-rbac-proxy" containerID="cri-o://c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4" gracePeriod=600 Aug 04 10:30:26.233872 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.233846 2535 generic.go:356] "Generic (PLEG): container finished" podID="16710393-39e5-40af-a484-0c0a881639fd" containerID="cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b" exitCode=0 Aug 04 10:30:26.233968 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.233918 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16710393-39e5-40af-a484-0c0a881639fd","Type":"ContainerDied","Data":"cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b"} Aug 04 10:30:26.371517 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.371441 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:26.538706 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.538671 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-configmap-serving-certs-ca-bundle\") pod \"16710393-39e5-40af-a484-0c0a881639fd\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " Aug 04 10:30:26.538706 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.538714 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"16710393-39e5-40af-a484-0c0a881639fd\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " Aug 04 10:30:26.538949 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.538732 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-prometheus-k8s-rulefiles-0\") pod \"16710393-39e5-40af-a484-0c0a881639fd\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " Aug 04 10:30:26.538949 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.538756 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"16710393-39e5-40af-a484-0c0a881639fd\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " Aug 04 10:30:26.538949 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.538849 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-metrics-client-certs\") pod \"16710393-39e5-40af-a484-0c0a881639fd\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " Aug 04 10:30:26.538949 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.538880 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-prometheus-trusted-ca-bundle\") pod \"16710393-39e5-40af-a484-0c0a881639fd\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " Aug 04 10:30:26.538949 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.538902 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-thanos-prometheus-http-client-file\") pod \"16710393-39e5-40af-a484-0c0a881639fd\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " Aug 04 10:30:26.538949 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.538950 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/16710393-39e5-40af-a484-0c0a881639fd-prometheus-k8s-db\") pod \"16710393-39e5-40af-a484-0c0a881639fd\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " Aug 04 10:30:26.539236 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.538984 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2z858\" (UniqueName: \"kubernetes.io/projected/16710393-39e5-40af-a484-0c0a881639fd-kube-api-access-2z858\") pod \"16710393-39e5-40af-a484-0c0a881639fd\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " Aug 04 10:30:26.539236 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.539019 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-web-config\") pod \"16710393-39e5-40af-a484-0c0a881639fd\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " Aug 04 10:30:26.539236 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.539048 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-grpc-tls\") pod \"16710393-39e5-40af-a484-0c0a881639fd\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " Aug 04 10:30:26.539236 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.539079 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-prometheus-k8s-tls\") pod \"16710393-39e5-40af-a484-0c0a881639fd\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " Aug 04 10:30:26.539236 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.539107 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-prometheus-k8s-rulefiles-2\") pod \"16710393-39e5-40af-a484-0c0a881639fd\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " Aug 04 10:30:26.539236 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.539147 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-prometheus-k8s-rulefiles-1\") pod \"16710393-39e5-40af-a484-0c0a881639fd\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " Aug 04 10:30:26.539236 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.539173 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-kube-rbac-proxy\") pod \"16710393-39e5-40af-a484-0c0a881639fd\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " Aug 04 10:30:26.539236 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.539204 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/16710393-39e5-40af-a484-0c0a881639fd-tls-assets\") pod \"16710393-39e5-40af-a484-0c0a881639fd\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " Aug 04 10:30:26.539236 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.539205 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "16710393-39e5-40af-a484-0c0a881639fd" (UID: "16710393-39e5-40af-a484-0c0a881639fd"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:30:26.539236 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.539229 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-configmap-metrics-client-ca\") pod \"16710393-39e5-40af-a484-0c0a881639fd\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " Aug 04 10:30:26.539758 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.539282 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/16710393-39e5-40af-a484-0c0a881639fd-config-out\") pod \"16710393-39e5-40af-a484-0c0a881639fd\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " Aug 04 10:30:26.539758 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.539308 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-configmap-kubelet-serving-ca-bundle\") pod \"16710393-39e5-40af-a484-0c0a881639fd\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " Aug 04 10:30:26.539758 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.539337 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-config\") pod \"16710393-39e5-40af-a484-0c0a881639fd\" (UID: \"16710393-39e5-40af-a484-0c0a881639fd\") " Aug 04 10:30:26.539758 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.539601 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "16710393-39e5-40af-a484-0c0a881639fd" (UID: "16710393-39e5-40af-a484-0c0a881639fd"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:30:26.539758 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.539634 2535 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:26.540315 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.540285 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "16710393-39e5-40af-a484-0c0a881639fd" (UID: "16710393-39e5-40af-a484-0c0a881639fd"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:30:26.540895 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.540781 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16710393-39e5-40af-a484-0c0a881639fd-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "16710393-39e5-40af-a484-0c0a881639fd" (UID: "16710393-39e5-40af-a484-0c0a881639fd"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:30:26.540895 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.540818 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "16710393-39e5-40af-a484-0c0a881639fd" (UID: "16710393-39e5-40af-a484-0c0a881639fd"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:30:26.541542 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.541519 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "16710393-39e5-40af-a484-0c0a881639fd" (UID: "16710393-39e5-40af-a484-0c0a881639fd"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:30:26.541629 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.541542 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "16710393-39e5-40af-a484-0c0a881639fd" (UID: "16710393-39e5-40af-a484-0c0a881639fd"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:30:26.541629 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.541551 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "16710393-39e5-40af-a484-0c0a881639fd" (UID: "16710393-39e5-40af-a484-0c0a881639fd"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:30:26.541629 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.541603 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "16710393-39e5-40af-a484-0c0a881639fd" (UID: "16710393-39e5-40af-a484-0c0a881639fd"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 10:30:26.541850 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.541826 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-config" (OuterVolumeSpecName: "config") pod "16710393-39e5-40af-a484-0c0a881639fd" (UID: "16710393-39e5-40af-a484-0c0a881639fd"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 10:30:26.542318 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.542108 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "16710393-39e5-40af-a484-0c0a881639fd" (UID: "16710393-39e5-40af-a484-0c0a881639fd"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 10:30:26.542318 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.542202 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "16710393-39e5-40af-a484-0c0a881639fd" (UID: "16710393-39e5-40af-a484-0c0a881639fd"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 10:30:26.542318 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.542257 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "16710393-39e5-40af-a484-0c0a881639fd" (UID: "16710393-39e5-40af-a484-0c0a881639fd"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 10:30:26.543110 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.543070 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "16710393-39e5-40af-a484-0c0a881639fd" (UID: "16710393-39e5-40af-a484-0c0a881639fd"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 10:30:26.543481 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.543462 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16710393-39e5-40af-a484-0c0a881639fd-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "16710393-39e5-40af-a484-0c0a881639fd" (UID: "16710393-39e5-40af-a484-0c0a881639fd"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:30:26.543666 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.543645 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "16710393-39e5-40af-a484-0c0a881639fd" (UID: "16710393-39e5-40af-a484-0c0a881639fd"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 10:30:26.543884 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.543860 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16710393-39e5-40af-a484-0c0a881639fd-config-out" (OuterVolumeSpecName: "config-out") pod "16710393-39e5-40af-a484-0c0a881639fd" (UID: "16710393-39e5-40af-a484-0c0a881639fd"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:30:26.543968 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.543950 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16710393-39e5-40af-a484-0c0a881639fd-kube-api-access-2z858" (OuterVolumeSpecName: "kube-api-access-2z858") pod "16710393-39e5-40af-a484-0c0a881639fd" (UID: "16710393-39e5-40af-a484-0c0a881639fd"). InnerVolumeSpecName "kube-api-access-2z858". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:30:26.544063 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.544047 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "16710393-39e5-40af-a484-0c0a881639fd" (UID: "16710393-39e5-40af-a484-0c0a881639fd"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 10:30:26.552528 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.552508 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-web-config" (OuterVolumeSpecName: "web-config") pod "16710393-39e5-40af-a484-0c0a881639fd" (UID: "16710393-39e5-40af-a484-0c0a881639fd"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 10:30:26.641057 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.640959 2535 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/16710393-39e5-40af-a484-0c0a881639fd-prometheus-k8s-db\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:26.641057 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.640998 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2z858\" (UniqueName: \"kubernetes.io/projected/16710393-39e5-40af-a484-0c0a881639fd-kube-api-access-2z858\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:26.641057 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.641012 2535 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-web-config\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:26.641057 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.641028 2535 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-grpc-tls\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:26.641057 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.641041 2535 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-prometheus-k8s-tls\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:26.641057 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.641055 2535 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:26.641442 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.641068 2535 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:26.641442 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.641081 2535 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-kube-rbac-proxy\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:26.641442 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.641097 2535 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/16710393-39e5-40af-a484-0c0a881639fd-tls-assets\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:26.641442 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.641110 2535 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-configmap-metrics-client-ca\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:26.641442 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.641123 2535 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/16710393-39e5-40af-a484-0c0a881639fd-config-out\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:26.641442 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.641140 2535 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:26.641442 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.641154 2535 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-config\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:26.641790 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.641168 2535 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:26.641790 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.641648 2535 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:26.641790 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.641674 2535 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:26.641790 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.641694 2535 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-secret-metrics-client-certs\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:26.641790 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.641718 2535 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16710393-39e5-40af-a484-0c0a881639fd-prometheus-trusted-ca-bundle\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:26.641790 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:26.641733 2535 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/16710393-39e5-40af-a484-0c0a881639fd-thanos-prometheus-http-client-file\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:30:27.241389 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.241349 2535 generic.go:356] "Generic (PLEG): container finished" podID="16710393-39e5-40af-a484-0c0a881639fd" containerID="34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe" exitCode=0 Aug 04 10:30:27.241389 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.241375 2535 generic.go:356] "Generic (PLEG): container finished" podID="16710393-39e5-40af-a484-0c0a881639fd" containerID="c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4" exitCode=0 Aug 04 10:30:27.241389 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.241382 2535 generic.go:356] "Generic (PLEG): container finished" podID="16710393-39e5-40af-a484-0c0a881639fd" containerID="9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770" exitCode=0 Aug 04 10:30:27.241389 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.241388 2535 generic.go:356] "Generic (PLEG): container finished" podID="16710393-39e5-40af-a484-0c0a881639fd" containerID="fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba" exitCode=0 Aug 04 10:30:27.241389 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.241393 2535 generic.go:356] "Generic (PLEG): container finished" podID="16710393-39e5-40af-a484-0c0a881639fd" containerID="fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884" exitCode=0 Aug 04 10:30:27.242035 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.241431 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16710393-39e5-40af-a484-0c0a881639fd","Type":"ContainerDied","Data":"34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe"} Aug 04 10:30:27.242035 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.241447 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.242035 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.241548 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16710393-39e5-40af-a484-0c0a881639fd","Type":"ContainerDied","Data":"c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4"} Aug 04 10:30:27.242035 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.241581 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16710393-39e5-40af-a484-0c0a881639fd","Type":"ContainerDied","Data":"9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770"} Aug 04 10:30:27.242035 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.241606 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16710393-39e5-40af-a484-0c0a881639fd","Type":"ContainerDied","Data":"fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba"} Aug 04 10:30:27.242035 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.241622 2535 scope.go:117] "RemoveContainer" containerID="34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe" Aug 04 10:30:27.242035 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.241630 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16710393-39e5-40af-a484-0c0a881639fd","Type":"ContainerDied","Data":"fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884"} Aug 04 10:30:27.242035 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.241723 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"16710393-39e5-40af-a484-0c0a881639fd","Type":"ContainerDied","Data":"5247578cf3446a8758d590f18e0560f29c129881e6131b9316977600e6402f62"} Aug 04 10:30:27.249430 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.249410 2535 scope.go:117] "RemoveContainer" containerID="c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4" Aug 04 10:30:27.256070 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.256056 2535 scope.go:117] "RemoveContainer" containerID="cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b" Aug 04 10:30:27.262156 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.262135 2535 scope.go:117] "RemoveContainer" containerID="9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770" Aug 04 10:30:27.265133 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.265114 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 10:30:27.268856 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.268842 2535 scope.go:117] "RemoveContainer" containerID="fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba" Aug 04 10:30:27.271840 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.271818 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 10:30:27.275601 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.275583 2535 scope.go:117] "RemoveContainer" containerID="fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884" Aug 04 10:30:27.282157 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.282139 2535 scope.go:117] "RemoveContainer" containerID="f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63" Aug 04 10:30:27.289835 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.289815 2535 scope.go:117] "RemoveContainer" containerID="34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe" Aug 04 10:30:27.290444 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:30:27.290427 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe\": container with ID starting with 34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe not found: ID does not exist" containerID="34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe" Aug 04 10:30:27.290515 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.290452 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe"} err="failed to get container status \"34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe\": rpc error: code = NotFound desc = could not find container \"34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe\": container with ID starting with 34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe not found: ID does not exist" Aug 04 10:30:27.290515 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.290470 2535 scope.go:117] "RemoveContainer" containerID="c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4" Aug 04 10:30:27.290773 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:30:27.290754 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4\": container with ID starting with c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4 not found: ID does not exist" containerID="c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4" Aug 04 10:30:27.290849 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.290781 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4"} err="failed to get container status \"c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4\": rpc error: code = NotFound desc = could not find container \"c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4\": container with ID starting with c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4 not found: ID does not exist" Aug 04 10:30:27.290849 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.290805 2535 scope.go:117] "RemoveContainer" containerID="cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b" Aug 04 10:30:27.291035 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:30:27.291020 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b\": container with ID starting with cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b not found: ID does not exist" containerID="cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b" Aug 04 10:30:27.291095 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.291042 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b"} err="failed to get container status \"cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b\": rpc error: code = NotFound desc = could not find container \"cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b\": container with ID starting with cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b not found: ID does not exist" Aug 04 10:30:27.291095 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.291061 2535 scope.go:117] "RemoveContainer" containerID="9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770" Aug 04 10:30:27.291388 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:30:27.291348 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770\": container with ID starting with 9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770 not found: ID does not exist" containerID="9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770" Aug 04 10:30:27.291463 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.291420 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770"} err="failed to get container status \"9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770\": rpc error: code = NotFound desc = could not find container \"9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770\": container with ID starting with 9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770 not found: ID does not exist" Aug 04 10:30:27.291463 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.291449 2535 scope.go:117] "RemoveContainer" containerID="fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba" Aug 04 10:30:27.291881 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:30:27.291772 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba\": container with ID starting with fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba not found: ID does not exist" containerID="fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba" Aug 04 10:30:27.291881 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.291802 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba"} err="failed to get container status \"fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba\": rpc error: code = NotFound desc = could not find container \"fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba\": container with ID starting with fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba not found: ID does not exist" Aug 04 10:30:27.291881 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.291822 2535 scope.go:117] "RemoveContainer" containerID="fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884" Aug 04 10:30:27.292171 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:30:27.292084 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884\": container with ID starting with fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884 not found: ID does not exist" containerID="fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884" Aug 04 10:30:27.292171 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.292105 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884"} err="failed to get container status \"fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884\": rpc error: code = NotFound desc = could not find container \"fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884\": container with ID starting with fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884 not found: ID does not exist" Aug 04 10:30:27.292171 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.292124 2535 scope.go:117] "RemoveContainer" containerID="f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63" Aug 04 10:30:27.292386 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:30:27.292370 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63\": container with ID starting with f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63 not found: ID does not exist" containerID="f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63" Aug 04 10:30:27.292428 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.292389 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63"} err="failed to get container status \"f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63\": rpc error: code = NotFound desc = could not find container \"f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63\": container with ID starting with f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63 not found: ID does not exist" Aug 04 10:30:27.292428 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.292403 2535 scope.go:117] "RemoveContainer" containerID="34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe" Aug 04 10:30:27.292657 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.292635 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe"} err="failed to get container status \"34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe\": rpc error: code = NotFound desc = could not find container \"34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe\": container with ID starting with 34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe not found: ID does not exist" Aug 04 10:30:27.292708 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.292657 2535 scope.go:117] "RemoveContainer" containerID="c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4" Aug 04 10:30:27.292883 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.292864 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4"} err="failed to get container status \"c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4\": rpc error: code = NotFound desc = could not find container \"c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4\": container with ID starting with c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4 not found: ID does not exist" Aug 04 10:30:27.292920 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.292884 2535 scope.go:117] "RemoveContainer" containerID="cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b" Aug 04 10:30:27.293095 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.293076 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b"} err="failed to get container status \"cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b\": rpc error: code = NotFound desc = could not find container \"cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b\": container with ID starting with cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b not found: ID does not exist" Aug 04 10:30:27.293136 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.293097 2535 scope.go:117] "RemoveContainer" containerID="9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770" Aug 04 10:30:27.293316 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.293299 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770"} err="failed to get container status \"9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770\": rpc error: code = NotFound desc = could not find container \"9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770\": container with ID starting with 9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770 not found: ID does not exist" Aug 04 10:30:27.293366 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.293317 2535 scope.go:117] "RemoveContainer" containerID="fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba" Aug 04 10:30:27.293547 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.293532 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba"} err="failed to get container status \"fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba\": rpc error: code = NotFound desc = could not find container \"fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba\": container with ID starting with fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba not found: ID does not exist" Aug 04 10:30:27.293597 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.293547 2535 scope.go:117] "RemoveContainer" containerID="fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884" Aug 04 10:30:27.293761 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.293743 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884"} err="failed to get container status \"fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884\": rpc error: code = NotFound desc = could not find container \"fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884\": container with ID starting with fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884 not found: ID does not exist" Aug 04 10:30:27.293803 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.293762 2535 scope.go:117] "RemoveContainer" containerID="f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63" Aug 04 10:30:27.293947 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.293932 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63"} err="failed to get container status \"f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63\": rpc error: code = NotFound desc = could not find container \"f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63\": container with ID starting with f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63 not found: ID does not exist" Aug 04 10:30:27.293947 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.293946 2535 scope.go:117] "RemoveContainer" containerID="34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe" Aug 04 10:30:27.294127 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.294109 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe"} err="failed to get container status \"34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe\": rpc error: code = NotFound desc = could not find container \"34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe\": container with ID starting with 34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe not found: ID does not exist" Aug 04 10:30:27.294164 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.294127 2535 scope.go:117] "RemoveContainer" containerID="c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4" Aug 04 10:30:27.294299 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.294284 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4"} err="failed to get container status \"c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4\": rpc error: code = NotFound desc = could not find container \"c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4\": container with ID starting with c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4 not found: ID does not exist" Aug 04 10:30:27.294335 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.294299 2535 scope.go:117] "RemoveContainer" containerID="cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b" Aug 04 10:30:27.294466 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.294451 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b"} err="failed to get container status \"cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b\": rpc error: code = NotFound desc = could not find container \"cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b\": container with ID starting with cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b not found: ID does not exist" Aug 04 10:30:27.294532 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.294466 2535 scope.go:117] "RemoveContainer" containerID="9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770" Aug 04 10:30:27.294683 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.294668 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770"} err="failed to get container status \"9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770\": rpc error: code = NotFound desc = could not find container \"9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770\": container with ID starting with 9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770 not found: ID does not exist" Aug 04 10:30:27.294726 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.294683 2535 scope.go:117] "RemoveContainer" containerID="fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba" Aug 04 10:30:27.294854 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.294833 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba"} err="failed to get container status \"fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba\": rpc error: code = NotFound desc = could not find container \"fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba\": container with ID starting with fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba not found: ID does not exist" Aug 04 10:30:27.294895 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.294855 2535 scope.go:117] "RemoveContainer" containerID="fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884" Aug 04 10:30:27.295017 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.295002 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884"} err="failed to get container status \"fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884\": rpc error: code = NotFound desc = could not find container \"fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884\": container with ID starting with fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884 not found: ID does not exist" Aug 04 10:30:27.295017 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.295016 2535 scope.go:117] "RemoveContainer" containerID="f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63" Aug 04 10:30:27.295184 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.295168 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63"} err="failed to get container status \"f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63\": rpc error: code = NotFound desc = could not find container \"f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63\": container with ID starting with f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63 not found: ID does not exist" Aug 04 10:30:27.295223 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.295183 2535 scope.go:117] "RemoveContainer" containerID="34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe" Aug 04 10:30:27.295342 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.295328 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe"} err="failed to get container status \"34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe\": rpc error: code = NotFound desc = could not find container \"34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe\": container with ID starting with 34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe not found: ID does not exist" Aug 04 10:30:27.295377 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.295343 2535 scope.go:117] "RemoveContainer" containerID="c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4" Aug 04 10:30:27.295534 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.295518 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4"} err="failed to get container status \"c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4\": rpc error: code = NotFound desc = could not find container \"c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4\": container with ID starting with c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4 not found: ID does not exist" Aug 04 10:30:27.295580 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.295534 2535 scope.go:117] "RemoveContainer" containerID="cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b" Aug 04 10:30:27.295715 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.295699 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b"} err="failed to get container status \"cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b\": rpc error: code = NotFound desc = could not find container \"cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b\": container with ID starting with cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b not found: ID does not exist" Aug 04 10:30:27.295754 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.295715 2535 scope.go:117] "RemoveContainer" containerID="9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770" Aug 04 10:30:27.295901 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.295883 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770"} err="failed to get container status \"9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770\": rpc error: code = NotFound desc = could not find container \"9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770\": container with ID starting with 9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770 not found: ID does not exist" Aug 04 10:30:27.295936 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.295902 2535 scope.go:117] "RemoveContainer" containerID="fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba" Aug 04 10:30:27.296113 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.296099 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba"} err="failed to get container status \"fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba\": rpc error: code = NotFound desc = could not find container \"fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba\": container with ID starting with fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba not found: ID does not exist" Aug 04 10:30:27.296113 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.296112 2535 scope.go:117] "RemoveContainer" containerID="fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884" Aug 04 10:30:27.296310 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.296294 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884"} err="failed to get container status \"fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884\": rpc error: code = NotFound desc = could not find container \"fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884\": container with ID starting with fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884 not found: ID does not exist" Aug 04 10:30:27.296351 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.296310 2535 scope.go:117] "RemoveContainer" containerID="f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63" Aug 04 10:30:27.296498 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.296472 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63"} err="failed to get container status \"f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63\": rpc error: code = NotFound desc = could not find container \"f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63\": container with ID starting with f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63 not found: ID does not exist" Aug 04 10:30:27.296543 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.296503 2535 scope.go:117] "RemoveContainer" containerID="34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe" Aug 04 10:30:27.296684 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.296665 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe"} err="failed to get container status \"34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe\": rpc error: code = NotFound desc = could not find container \"34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe\": container with ID starting with 34731769dc6451fb0cf302fa335873db36fa0c7c67d124c5beb9ce07a8b8efbe not found: ID does not exist" Aug 04 10:30:27.296724 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.296684 2535 scope.go:117] "RemoveContainer" containerID="c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4" Aug 04 10:30:27.296855 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.296841 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4"} err="failed to get container status \"c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4\": rpc error: code = NotFound desc = could not find container \"c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4\": container with ID starting with c29e4c1a6a5329dc4d719a7521d04cb1ab229e1177ad62f5765223d0e91b0ea4 not found: ID does not exist" Aug 04 10:30:27.296898 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.296854 2535 scope.go:117] "RemoveContainer" containerID="cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b" Aug 04 10:30:27.297021 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.297002 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b"} err="failed to get container status \"cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b\": rpc error: code = NotFound desc = could not find container \"cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b\": container with ID starting with cea605e241a021d8593eabb1d74b1467f1ecabda443eec9e351fc44e8931da1b not found: ID does not exist" Aug 04 10:30:27.297061 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.297021 2535 scope.go:117] "RemoveContainer" containerID="9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770" Aug 04 10:30:27.297178 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.297164 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770"} err="failed to get container status \"9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770\": rpc error: code = NotFound desc = could not find container \"9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770\": container with ID starting with 9d03b45f6f1a3230544dc4fc322408f1608ab385f8290650458dd90541404770 not found: ID does not exist" Aug 04 10:30:27.297178 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.297177 2535 scope.go:117] "RemoveContainer" containerID="fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba" Aug 04 10:30:27.297391 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.297361 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba"} err="failed to get container status \"fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba\": rpc error: code = NotFound desc = could not find container \"fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba\": container with ID starting with fe155a9e8e6a6c241083c14f011aa20ee8dcb1b60c0adeab26d8b19d8ff262ba not found: ID does not exist" Aug 04 10:30:27.297460 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.297391 2535 scope.go:117] "RemoveContainer" containerID="fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884" Aug 04 10:30:27.297662 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.297644 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884"} err="failed to get container status \"fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884\": rpc error: code = NotFound desc = could not find container \"fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884\": container with ID starting with fcda87b3c14de2e497c63b21a48787faa3535df064b4d73a8a5471b2a3221884 not found: ID does not exist" Aug 04 10:30:27.297722 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.297661 2535 scope.go:117] "RemoveContainer" containerID="f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63" Aug 04 10:30:27.297941 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.297916 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63"} err="failed to get container status \"f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63\": rpc error: code = NotFound desc = could not find container \"f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63\": container with ID starting with f472ad891147262338722d144fc78401aba3be12106ebfa45a88c628c6495b63 not found: ID does not exist" Aug 04 10:30:27.302825 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.302805 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 10:30:27.303058 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303047 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="220ac383-255d-4ddc-b2cf-957118bad51b" containerName="kube-rbac-proxy" Aug 04 10:30:27.303098 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303061 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="220ac383-255d-4ddc-b2cf-957118bad51b" containerName="kube-rbac-proxy" Aug 04 10:30:27.303098 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303070 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16710393-39e5-40af-a484-0c0a881639fd" containerName="kube-rbac-proxy-web" Aug 04 10:30:27.303098 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303075 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="16710393-39e5-40af-a484-0c0a881639fd" containerName="kube-rbac-proxy-web" Aug 04 10:30:27.303098 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303082 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16710393-39e5-40af-a484-0c0a881639fd" containerName="kube-rbac-proxy" Aug 04 10:30:27.303098 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303088 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="16710393-39e5-40af-a484-0c0a881639fd" containerName="kube-rbac-proxy" Aug 04 10:30:27.303098 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303096 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16710393-39e5-40af-a484-0c0a881639fd" containerName="prometheus" Aug 04 10:30:27.303098 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303101 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="16710393-39e5-40af-a484-0c0a881639fd" containerName="prometheus" Aug 04 10:30:27.303287 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303107 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16710393-39e5-40af-a484-0c0a881639fd" containerName="config-reloader" Aug 04 10:30:27.303287 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303112 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="16710393-39e5-40af-a484-0c0a881639fd" containerName="config-reloader" Aug 04 10:30:27.303287 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303120 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="220ac383-255d-4ddc-b2cf-957118bad51b" containerName="kube-rbac-proxy-metric" Aug 04 10:30:27.303287 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303126 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="220ac383-255d-4ddc-b2cf-957118bad51b" containerName="kube-rbac-proxy-metric" Aug 04 10:30:27.303287 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303134 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="220ac383-255d-4ddc-b2cf-957118bad51b" containerName="prom-label-proxy" Aug 04 10:30:27.303287 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303139 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="220ac383-255d-4ddc-b2cf-957118bad51b" containerName="prom-label-proxy" Aug 04 10:30:27.303287 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303146 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16710393-39e5-40af-a484-0c0a881639fd" containerName="kube-rbac-proxy-thanos" Aug 04 10:30:27.303287 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303151 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="16710393-39e5-40af-a484-0c0a881639fd" containerName="kube-rbac-proxy-thanos" Aug 04 10:30:27.303287 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303159 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="220ac383-255d-4ddc-b2cf-957118bad51b" containerName="init-config-reloader" Aug 04 10:30:27.303287 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303165 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="220ac383-255d-4ddc-b2cf-957118bad51b" containerName="init-config-reloader" Aug 04 10:30:27.303287 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303171 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="220ac383-255d-4ddc-b2cf-957118bad51b" containerName="alertmanager" Aug 04 10:30:27.303287 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303175 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="220ac383-255d-4ddc-b2cf-957118bad51b" containerName="alertmanager" Aug 04 10:30:27.303287 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303180 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="220ac383-255d-4ddc-b2cf-957118bad51b" containerName="config-reloader" Aug 04 10:30:27.303287 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303185 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="220ac383-255d-4ddc-b2cf-957118bad51b" containerName="config-reloader" Aug 04 10:30:27.303287 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303191 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16710393-39e5-40af-a484-0c0a881639fd" containerName="thanos-sidecar" Aug 04 10:30:27.303287 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303196 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="16710393-39e5-40af-a484-0c0a881639fd" containerName="thanos-sidecar" Aug 04 10:30:27.303287 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303201 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="220ac383-255d-4ddc-b2cf-957118bad51b" containerName="kube-rbac-proxy-web" Aug 04 10:30:27.303287 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303206 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="220ac383-255d-4ddc-b2cf-957118bad51b" containerName="kube-rbac-proxy-web" Aug 04 10:30:27.303287 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303212 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16710393-39e5-40af-a484-0c0a881639fd" containerName="init-config-reloader" Aug 04 10:30:27.303287 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303217 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="16710393-39e5-40af-a484-0c0a881639fd" containerName="init-config-reloader" Aug 04 10:30:27.303287 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303254 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="220ac383-255d-4ddc-b2cf-957118bad51b" containerName="alertmanager" Aug 04 10:30:27.303287 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303260 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="16710393-39e5-40af-a484-0c0a881639fd" containerName="config-reloader" Aug 04 10:30:27.303287 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303266 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="16710393-39e5-40af-a484-0c0a881639fd" containerName="thanos-sidecar" Aug 04 10:30:27.303287 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303271 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="16710393-39e5-40af-a484-0c0a881639fd" containerName="kube-rbac-proxy-web" Aug 04 10:30:27.303287 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303277 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="220ac383-255d-4ddc-b2cf-957118bad51b" containerName="config-reloader" Aug 04 10:30:27.303287 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303283 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="16710393-39e5-40af-a484-0c0a881639fd" containerName="prometheus" Aug 04 10:30:27.303287 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303287 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="16710393-39e5-40af-a484-0c0a881639fd" containerName="kube-rbac-proxy" Aug 04 10:30:27.303287 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303292 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="16710393-39e5-40af-a484-0c0a881639fd" containerName="kube-rbac-proxy-thanos" Aug 04 10:30:27.303287 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303297 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="220ac383-255d-4ddc-b2cf-957118bad51b" containerName="kube-rbac-proxy-web" Aug 04 10:30:27.304145 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303303 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="220ac383-255d-4ddc-b2cf-957118bad51b" containerName="kube-rbac-proxy" Aug 04 10:30:27.304145 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303308 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="220ac383-255d-4ddc-b2cf-957118bad51b" containerName="kube-rbac-proxy-metric" Aug 04 10:30:27.304145 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.303312 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="220ac383-255d-4ddc-b2cf-957118bad51b" containerName="prom-label-proxy" Aug 04 10:30:27.309516 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.309500 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.311730 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.311710 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Aug 04 10:30:27.311976 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.311961 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Aug 04 10:30:27.312054 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.311975 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Aug 04 10:30:27.312054 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.311962 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Aug 04 10:30:27.312141 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.311962 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-5vpl2\"" Aug 04 10:30:27.312328 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.312312 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Aug 04 10:30:27.312505 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.312472 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Aug 04 10:30:27.313068 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.313052 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Aug 04 10:30:27.313117 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.313079 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Aug 04 10:30:27.313117 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.313059 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Aug 04 10:30:27.313117 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.313102 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Aug 04 10:30:27.313745 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.313729 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Aug 04 10:30:27.313900 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.313886 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Aug 04 10:30:27.314167 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.314151 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Aug 04 10:30:27.314332 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.314319 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-57s78eprjclpa\"" Aug 04 10:30:27.316181 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.316148 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Aug 04 10:30:27.318474 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.318457 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Aug 04 10:30:27.324760 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.324740 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 10:30:27.449024 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.448940 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2be47b9e-0e11-4bc8-9608-158357a2f2d2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.449024 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.448984 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2be47b9e-0e11-4bc8-9608-158357a2f2d2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.449024 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.449003 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2be47b9e-0e11-4bc8-9608-158357a2f2d2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.449300 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.449093 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2be47b9e-0e11-4bc8-9608-158357a2f2d2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.449300 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.449138 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2be47b9e-0e11-4bc8-9608-158357a2f2d2-web-config\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.449300 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.449174 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2be47b9e-0e11-4bc8-9608-158357a2f2d2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.449300 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.449196 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2be47b9e-0e11-4bc8-9608-158357a2f2d2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.449300 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.449224 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2be47b9e-0e11-4bc8-9608-158357a2f2d2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.449300 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.449248 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcp85\" (UniqueName: \"kubernetes.io/projected/2be47b9e-0e11-4bc8-9608-158357a2f2d2-kube-api-access-gcp85\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.449594 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.449292 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2be47b9e-0e11-4bc8-9608-158357a2f2d2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.449594 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.449349 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2be47b9e-0e11-4bc8-9608-158357a2f2d2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.449594 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.449380 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2be47b9e-0e11-4bc8-9608-158357a2f2d2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.449594 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.449434 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2be47b9e-0e11-4bc8-9608-158357a2f2d2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.449594 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.449479 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2be47b9e-0e11-4bc8-9608-158357a2f2d2-config-out\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.449594 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.449530 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2be47b9e-0e11-4bc8-9608-158357a2f2d2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.449594 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.449563 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2be47b9e-0e11-4bc8-9608-158357a2f2d2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.449594 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.449588 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2be47b9e-0e11-4bc8-9608-158357a2f2d2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.449869 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.449609 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2be47b9e-0e11-4bc8-9608-158357a2f2d2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.449869 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.449649 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2be47b9e-0e11-4bc8-9608-158357a2f2d2-config\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.449869 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.449666 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2be47b9e-0e11-4bc8-9608-158357a2f2d2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.550285 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.550247 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2be47b9e-0e11-4bc8-9608-158357a2f2d2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.550285 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.550285 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2be47b9e-0e11-4bc8-9608-158357a2f2d2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.550544 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.550312 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2be47b9e-0e11-4bc8-9608-158357a2f2d2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.550544 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.550335 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2be47b9e-0e11-4bc8-9608-158357a2f2d2-web-config\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.550544 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.550364 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2be47b9e-0e11-4bc8-9608-158357a2f2d2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.550544 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.550385 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2be47b9e-0e11-4bc8-9608-158357a2f2d2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.550544 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.550409 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2be47b9e-0e11-4bc8-9608-158357a2f2d2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.550544 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.550434 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcp85\" (UniqueName: \"kubernetes.io/projected/2be47b9e-0e11-4bc8-9608-158357a2f2d2-kube-api-access-gcp85\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.550544 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.550460 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2be47b9e-0e11-4bc8-9608-158357a2f2d2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.550544 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.550519 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2be47b9e-0e11-4bc8-9608-158357a2f2d2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.550932 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.550554 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2be47b9e-0e11-4bc8-9608-158357a2f2d2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.550932 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.550593 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2be47b9e-0e11-4bc8-9608-158357a2f2d2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.550932 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.550657 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2be47b9e-0e11-4bc8-9608-158357a2f2d2-config-out\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.550932 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.550682 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2be47b9e-0e11-4bc8-9608-158357a2f2d2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.550932 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.550704 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2be47b9e-0e11-4bc8-9608-158357a2f2d2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.550932 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.550732 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2be47b9e-0e11-4bc8-9608-158357a2f2d2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.550932 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.550774 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2be47b9e-0e11-4bc8-9608-158357a2f2d2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.550932 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.550808 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2be47b9e-0e11-4bc8-9608-158357a2f2d2-config\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.550932 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.550829 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2be47b9e-0e11-4bc8-9608-158357a2f2d2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.551345 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.551080 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2be47b9e-0e11-4bc8-9608-158357a2f2d2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.551345 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.551097 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2be47b9e-0e11-4bc8-9608-158357a2f2d2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.551345 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.551242 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2be47b9e-0e11-4bc8-9608-158357a2f2d2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.553281 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.553249 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2be47b9e-0e11-4bc8-9608-158357a2f2d2-web-config\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.553998 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.553666 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2be47b9e-0e11-4bc8-9608-158357a2f2d2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.553998 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.553840 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2be47b9e-0e11-4bc8-9608-158357a2f2d2-config\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.553998 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.553858 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2be47b9e-0e11-4bc8-9608-158357a2f2d2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.553998 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.550837 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2be47b9e-0e11-4bc8-9608-158357a2f2d2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.553998 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.553959 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2be47b9e-0e11-4bc8-9608-158357a2f2d2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.553998 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.553981 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2be47b9e-0e11-4bc8-9608-158357a2f2d2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.554350 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.554017 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2be47b9e-0e11-4bc8-9608-158357a2f2d2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.554404 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.554371 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2be47b9e-0e11-4bc8-9608-158357a2f2d2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.554934 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.554907 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2be47b9e-0e11-4bc8-9608-158357a2f2d2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.555033 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.554985 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2be47b9e-0e11-4bc8-9608-158357a2f2d2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.555381 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.555357 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2be47b9e-0e11-4bc8-9608-158357a2f2d2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.555801 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.555776 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2be47b9e-0e11-4bc8-9608-158357a2f2d2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.556231 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.556208 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2be47b9e-0e11-4bc8-9608-158357a2f2d2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.556390 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.556369 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2be47b9e-0e11-4bc8-9608-158357a2f2d2-config-out\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.556590 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.556574 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2be47b9e-0e11-4bc8-9608-158357a2f2d2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.557071 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.557048 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2be47b9e-0e11-4bc8-9608-158357a2f2d2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.558319 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.558298 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcp85\" (UniqueName: \"kubernetes.io/projected/2be47b9e-0e11-4bc8-9608-158357a2f2d2-kube-api-access-gcp85\") pod \"prometheus-k8s-0\" (UID: \"2be47b9e-0e11-4bc8-9608-158357a2f2d2\") " pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.619793 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.619753 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:27.762409 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:27.762381 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Aug 04 10:30:27.764803 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:30:27.764777 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2be47b9e_0e11_4bc8_9608_158357a2f2d2.slice/crio-52a791fc1347c1db76157f381dbe4ae7966b35757d41f2ac4fc19fcbed7e90f3 WatchSource:0}: Error finding container 52a791fc1347c1db76157f381dbe4ae7966b35757d41f2ac4fc19fcbed7e90f3: Status 404 returned error can't find the container with id 52a791fc1347c1db76157f381dbe4ae7966b35757d41f2ac4fc19fcbed7e90f3 Aug 04 10:30:28.248088 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:28.248058 2535 generic.go:356] "Generic (PLEG): container finished" podID="2be47b9e-0e11-4bc8-9608-158357a2f2d2" containerID="9bbdd7e9c1ee1a456213572e05631769071c0510aaf894b45dcfcca3bc8214e9" exitCode=0 Aug 04 10:30:28.248574 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:28.248107 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2be47b9e-0e11-4bc8-9608-158357a2f2d2","Type":"ContainerDied","Data":"9bbdd7e9c1ee1a456213572e05631769071c0510aaf894b45dcfcca3bc8214e9"} Aug 04 10:30:28.248574 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:28.248162 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2be47b9e-0e11-4bc8-9608-158357a2f2d2","Type":"ContainerStarted","Data":"52a791fc1347c1db76157f381dbe4ae7966b35757d41f2ac4fc19fcbed7e90f3"} Aug 04 10:30:28.482375 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:28.482334 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16710393-39e5-40af-a484-0c0a881639fd" path="/var/lib/kubelet/pods/16710393-39e5-40af-a484-0c0a881639fd/volumes" Aug 04 10:30:29.255637 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:29.255603 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2be47b9e-0e11-4bc8-9608-158357a2f2d2","Type":"ContainerStarted","Data":"c4439f3f134a19d6586f0b3ee4f0fcdbfd57582a27b207a9589351992dbe09ab"} Aug 04 10:30:29.256049 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:29.255665 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2be47b9e-0e11-4bc8-9608-158357a2f2d2","Type":"ContainerStarted","Data":"9f806fc150bd309492dab4faad21513f252e4c496e0e4275b23a401592edfde0"} Aug 04 10:30:29.256049 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:29.255692 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2be47b9e-0e11-4bc8-9608-158357a2f2d2","Type":"ContainerStarted","Data":"82a8dff22a7412f653965ec7f06a49720d73c7a53173cbf56424e27b4175fec1"} Aug 04 10:30:29.256049 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:29.255717 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2be47b9e-0e11-4bc8-9608-158357a2f2d2","Type":"ContainerStarted","Data":"669cc5af3986f4ad0a2cb6069663193dae333bfa9e179014cae8bad8a343c3d6"} Aug 04 10:30:29.256049 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:29.255740 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2be47b9e-0e11-4bc8-9608-158357a2f2d2","Type":"ContainerStarted","Data":"e4c8699d6d84e0e714a461b1947adc19bc1d87cd9d835c312776f640c968d439"} Aug 04 10:30:29.256049 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:29.255764 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2be47b9e-0e11-4bc8-9608-158357a2f2d2","Type":"ContainerStarted","Data":"40bf334a68cdb4c7397e2802593f6b785a6ca4dea2a8a157e4341349d8d949a1"} Aug 04 10:30:29.285178 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:29.285104 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.266496758 podStartE2EDuration="2.28508593s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.589172ms" isStatefulPod=false podCreationTimestamp="2026-08-04 10:30:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:30:29.283215104 +0000 UTC m=+167.257297649" watchObservedRunningTime="2026-08-04 10:30:29.28508593 +0000 UTC m=+167.259168456" Aug 04 10:30:29.452150 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:29.452120 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:30:29.516369 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:29.516264 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:30:32.620910 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:32.620856 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:30:35.621032 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:35.620959 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:30:36.308288 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:36.308248 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:30:36.409253 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:36.409223 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:30:54.589011 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:54.588969 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:30:54.645597 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:54.645563 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:30:54.737873 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:54.737837 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:30:55.330338 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:55.330308 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:30:55.429262 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:30:55.429231 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:31:05.802501 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:05.802439 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:31:05.902470 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:05.902438 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:31:05.961825 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:05.961799 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:31:07.283890 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:07.283852 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:31:07.344296 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:07.344259 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:31:20.848787 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:20.848749 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:31:20.946850 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:20.946760 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:31:25.249139 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:25.249102 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:31:25.349184 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:25.349157 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:31:27.620286 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:27.620253 2535 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:31:27.635185 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:27.635163 2535 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:31:28.485514 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:28.485474 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Aug 04 10:31:30.102364 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:30.102329 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:31:35.074923 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:35.074887 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:31:42.743284 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:42.743250 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:31:43.550822 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:43.550791 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:31:46.371468 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:46.371423 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:31:46.469223 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:46.469191 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:31:47.959172 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:47.959139 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:31:48.059375 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:48.059343 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:31:48.245131 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:48.245048 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:31:48.334247 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:48.334218 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:31:50.405329 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:50.405297 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:31:50.783003 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:50.782972 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:31:53.158726 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:53.158689 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:31:53.219364 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:53.219330 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:31:55.253067 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:55.253029 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:31:55.353395 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:55.353364 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:31:57.565024 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:57.564995 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:31:59.563707 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:59.563671 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:31:59.663097 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:31:59.663067 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:01.811042 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:01.811004 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jqp9s"] Aug 04 10:32:01.813278 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:01.813259 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jqp9s" Aug 04 10:32:01.815461 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:01.815443 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Aug 04 10:32:01.815940 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:01.815923 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-vbjdq\"" Aug 04 10:32:01.816027 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:01.815949 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Aug 04 10:32:01.823970 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:01.823949 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jqp9s"] Aug 04 10:32:01.883801 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:01.883771 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5pqn\" (UniqueName: \"kubernetes.io/projected/abaa863c-5c3f-44c8-aab6-c04da3416463-kube-api-access-f5pqn\") pod \"openshift-lws-operator-fd8ccff4c-jqp9s\" (UID: \"abaa863c-5c3f-44c8-aab6-c04da3416463\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jqp9s" Aug 04 10:32:01.883932 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:01.883834 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/abaa863c-5c3f-44c8-aab6-c04da3416463-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jqp9s\" (UID: \"abaa863c-5c3f-44c8-aab6-c04da3416463\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jqp9s" Aug 04 10:32:01.985092 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:01.985055 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5pqn\" (UniqueName: \"kubernetes.io/projected/abaa863c-5c3f-44c8-aab6-c04da3416463-kube-api-access-f5pqn\") pod \"openshift-lws-operator-fd8ccff4c-jqp9s\" (UID: \"abaa863c-5c3f-44c8-aab6-c04da3416463\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jqp9s" Aug 04 10:32:01.985256 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:01.985129 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/abaa863c-5c3f-44c8-aab6-c04da3416463-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jqp9s\" (UID: \"abaa863c-5c3f-44c8-aab6-c04da3416463\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jqp9s" Aug 04 10:32:01.985543 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:01.985525 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/abaa863c-5c3f-44c8-aab6-c04da3416463-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jqp9s\" (UID: \"abaa863c-5c3f-44c8-aab6-c04da3416463\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jqp9s" Aug 04 10:32:01.993468 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:01.993438 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5pqn\" (UniqueName: \"kubernetes.io/projected/abaa863c-5c3f-44c8-aab6-c04da3416463-kube-api-access-f5pqn\") pod \"openshift-lws-operator-fd8ccff4c-jqp9s\" (UID: \"abaa863c-5c3f-44c8-aab6-c04da3416463\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jqp9s" Aug 04 10:32:02.135692 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:02.135606 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jqp9s" Aug 04 10:32:02.261799 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:02.261767 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jqp9s"] Aug 04 10:32:02.266054 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:32:02.266027 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podabaa863c_5c3f_44c8_aab6_c04da3416463.slice/crio-cb9665c46b5f2bda098535e57b3d81dd43234c043ebcae8c07a03af4138ae321 WatchSource:0}: Error finding container cb9665c46b5f2bda098535e57b3d81dd43234c043ebcae8c07a03af4138ae321: Status 404 returned error can't find the container with id cb9665c46b5f2bda098535e57b3d81dd43234c043ebcae8c07a03af4138ae321 Aug 04 10:32:02.585452 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:02.585419 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jqp9s" event={"ID":"abaa863c-5c3f-44c8-aab6-c04da3416463","Type":"ContainerStarted","Data":"cb9665c46b5f2bda098535e57b3d81dd43234c043ebcae8c07a03af4138ae321"} Aug 04 10:32:05.600270 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:05.600236 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jqp9s" event={"ID":"abaa863c-5c3f-44c8-aab6-c04da3416463","Type":"ContainerStarted","Data":"0bd08ed6766b531ef4bb9b4fadb515778a39231838550af1102f4f00e457a320"} Aug 04 10:32:05.617339 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:05.617295 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jqp9s" podStartSLOduration=2.144287427 podStartE2EDuration="4.617279365s" totalImagesPullingTime="2.472991938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:32:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:32:05.61669087 +0000 UTC m=+263.590773392" watchObservedRunningTime="2026-08-04 10:32:05.617279365 +0000 UTC m=+263.591361892" Aug 04 10:32:06.303318 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:06.303278 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:06.404475 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:06.404444 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:08.178084 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:08.178051 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:20.043006 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:20.042970 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:20.128254 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:20.128214 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:20.601564 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:20.601534 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:20.839749 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:20.839718 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:20.947840 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:20.947760 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:21.005437 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.005407 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-598d67d848-dr22z"] Aug 04 10:32:21.012124 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.012106 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-598d67d848-dr22z" Aug 04 10:32:21.016253 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.016227 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Aug 04 10:32:21.016356 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.016238 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Aug 04 10:32:21.016356 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.016327 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-hd694\"" Aug 04 10:32:21.016356 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.016338 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Aug 04 10:32:21.016619 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.016605 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Aug 04 10:32:21.029777 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.029757 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-598d67d848-dr22z"] Aug 04 10:32:21.118674 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.118642 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:21.138455 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.138431 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/492e0c27-9f41-4a7c-b4ca-f93d3bd05d86-manifests\") pod \"opendatahub-operator-controller-manager-598d67d848-dr22z\" (UID: \"492e0c27-9f41-4a7c-b4ca-f93d3bd05d86\") " pod="opendatahub/opendatahub-operator-controller-manager-598d67d848-dr22z" Aug 04 10:32:21.138604 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.138467 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/492e0c27-9f41-4a7c-b4ca-f93d3bd05d86-tmp\") pod \"opendatahub-operator-controller-manager-598d67d848-dr22z\" (UID: \"492e0c27-9f41-4a7c-b4ca-f93d3bd05d86\") " pod="opendatahub/opendatahub-operator-controller-manager-598d67d848-dr22z" Aug 04 10:32:21.138604 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.138520 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/492e0c27-9f41-4a7c-b4ca-f93d3bd05d86-apiservice-cert\") pod \"opendatahub-operator-controller-manager-598d67d848-dr22z\" (UID: \"492e0c27-9f41-4a7c-b4ca-f93d3bd05d86\") " pod="opendatahub/opendatahub-operator-controller-manager-598d67d848-dr22z" Aug 04 10:32:21.138604 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.138547 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57f8l\" (UniqueName: \"kubernetes.io/projected/492e0c27-9f41-4a7c-b4ca-f93d3bd05d86-kube-api-access-57f8l\") pod \"opendatahub-operator-controller-manager-598d67d848-dr22z\" (UID: \"492e0c27-9f41-4a7c-b4ca-f93d3bd05d86\") " pod="opendatahub/opendatahub-operator-controller-manager-598d67d848-dr22z" Aug 04 10:32:21.138604 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.138577 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/492e0c27-9f41-4a7c-b4ca-f93d3bd05d86-webhook-cert\") pod \"opendatahub-operator-controller-manager-598d67d848-dr22z\" (UID: \"492e0c27-9f41-4a7c-b4ca-f93d3bd05d86\") " pod="opendatahub/opendatahub-operator-controller-manager-598d67d848-dr22z" Aug 04 10:32:21.217809 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.217729 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:21.239882 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.239852 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/492e0c27-9f41-4a7c-b4ca-f93d3bd05d86-manifests\") pod \"opendatahub-operator-controller-manager-598d67d848-dr22z\" (UID: \"492e0c27-9f41-4a7c-b4ca-f93d3bd05d86\") " pod="opendatahub/opendatahub-operator-controller-manager-598d67d848-dr22z" Aug 04 10:32:21.240025 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.239892 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/492e0c27-9f41-4a7c-b4ca-f93d3bd05d86-tmp\") pod \"opendatahub-operator-controller-manager-598d67d848-dr22z\" (UID: \"492e0c27-9f41-4a7c-b4ca-f93d3bd05d86\") " pod="opendatahub/opendatahub-operator-controller-manager-598d67d848-dr22z" Aug 04 10:32:21.240025 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.239917 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/492e0c27-9f41-4a7c-b4ca-f93d3bd05d86-apiservice-cert\") pod \"opendatahub-operator-controller-manager-598d67d848-dr22z\" (UID: \"492e0c27-9f41-4a7c-b4ca-f93d3bd05d86\") " pod="opendatahub/opendatahub-operator-controller-manager-598d67d848-dr22z" Aug 04 10:32:21.240346 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.240035 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57f8l\" (UniqueName: \"kubernetes.io/projected/492e0c27-9f41-4a7c-b4ca-f93d3bd05d86-kube-api-access-57f8l\") pod \"opendatahub-operator-controller-manager-598d67d848-dr22z\" (UID: \"492e0c27-9f41-4a7c-b4ca-f93d3bd05d86\") " pod="opendatahub/opendatahub-operator-controller-manager-598d67d848-dr22z" Aug 04 10:32:21.240346 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.240070 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/492e0c27-9f41-4a7c-b4ca-f93d3bd05d86-webhook-cert\") pod \"opendatahub-operator-controller-manager-598d67d848-dr22z\" (UID: \"492e0c27-9f41-4a7c-b4ca-f93d3bd05d86\") " pod="opendatahub/opendatahub-operator-controller-manager-598d67d848-dr22z" Aug 04 10:32:21.240458 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.240392 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/492e0c27-9f41-4a7c-b4ca-f93d3bd05d86-manifests\") pod \"opendatahub-operator-controller-manager-598d67d848-dr22z\" (UID: \"492e0c27-9f41-4a7c-b4ca-f93d3bd05d86\") " pod="opendatahub/opendatahub-operator-controller-manager-598d67d848-dr22z" Aug 04 10:32:21.242308 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.242285 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/492e0c27-9f41-4a7c-b4ca-f93d3bd05d86-tmp\") pod \"opendatahub-operator-controller-manager-598d67d848-dr22z\" (UID: \"492e0c27-9f41-4a7c-b4ca-f93d3bd05d86\") " pod="opendatahub/opendatahub-operator-controller-manager-598d67d848-dr22z" Aug 04 10:32:21.242478 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.242458 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/492e0c27-9f41-4a7c-b4ca-f93d3bd05d86-apiservice-cert\") pod \"opendatahub-operator-controller-manager-598d67d848-dr22z\" (UID: \"492e0c27-9f41-4a7c-b4ca-f93d3bd05d86\") " pod="opendatahub/opendatahub-operator-controller-manager-598d67d848-dr22z" Aug 04 10:32:21.242630 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.242608 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/492e0c27-9f41-4a7c-b4ca-f93d3bd05d86-webhook-cert\") pod \"opendatahub-operator-controller-manager-598d67d848-dr22z\" (UID: \"492e0c27-9f41-4a7c-b4ca-f93d3bd05d86\") " pod="opendatahub/opendatahub-operator-controller-manager-598d67d848-dr22z" Aug 04 10:32:21.248399 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.248376 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57f8l\" (UniqueName: \"kubernetes.io/projected/492e0c27-9f41-4a7c-b4ca-f93d3bd05d86-kube-api-access-57f8l\") pod \"opendatahub-operator-controller-manager-598d67d848-dr22z\" (UID: \"492e0c27-9f41-4a7c-b4ca-f93d3bd05d86\") " pod="opendatahub/opendatahub-operator-controller-manager-598d67d848-dr22z" Aug 04 10:32:21.320986 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.320957 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:21.322925 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.322911 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-598d67d848-dr22z" Aug 04 10:32:21.418134 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.418079 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:21.454862 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.454383 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-598d67d848-dr22z"] Aug 04 10:32:21.457098 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:32:21.457065 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod492e0c27_9f41_4a7c_b4ca_f93d3bd05d86.slice/crio-27a5767a9f5543b5cc228aae54679ef7294b22149f9a8b7f849a5609b3517e2c WatchSource:0}: Error finding container 27a5767a9f5543b5cc228aae54679ef7294b22149f9a8b7f849a5609b3517e2c: Status 404 returned error can't find the container with id 27a5767a9f5543b5cc228aae54679ef7294b22149f9a8b7f849a5609b3517e2c Aug 04 10:32:21.527832 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.527798 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:21.621692 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.621661 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:21.659540 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:21.659509 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-598d67d848-dr22z" event={"ID":"492e0c27-9f41-4a7c-b4ca-f93d3bd05d86","Type":"ContainerStarted","Data":"27a5767a9f5543b5cc228aae54679ef7294b22149f9a8b7f849a5609b3517e2c"} Aug 04 10:32:24.158816 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:24.158777 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:24.259184 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:24.259151 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:24.675395 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:24.675349 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-598d67d848-dr22z" event={"ID":"492e0c27-9f41-4a7c-b4ca-f93d3bd05d86","Type":"ContainerStarted","Data":"33139b2519ffa9df13fd53922611a23a22df74b94dd897d3d9207f9d1e1f69aa"} Aug 04 10:32:25.680548 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:25.680517 2535 generic.go:356] "Generic (PLEG): container finished" podID="492e0c27-9f41-4a7c-b4ca-f93d3bd05d86" containerID="33139b2519ffa9df13fd53922611a23a22df74b94dd897d3d9207f9d1e1f69aa" exitCode=0 Aug 04 10:32:25.680925 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:25.680600 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-598d67d848-dr22z" event={"ID":"492e0c27-9f41-4a7c-b4ca-f93d3bd05d86","Type":"ContainerDied","Data":"33139b2519ffa9df13fd53922611a23a22df74b94dd897d3d9207f9d1e1f69aa"} Aug 04 10:32:26.687587 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:26.687541 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-598d67d848-dr22z" event={"ID":"492e0c27-9f41-4a7c-b4ca-f93d3bd05d86","Type":"ContainerStarted","Data":"501c8200a6467de892e6076cd5b8649e42d219b3f71ef1ec65d3ba569905077f"} Aug 04 10:32:26.688005 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:26.687704 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-598d67d848-dr22z" Aug 04 10:32:26.708582 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:26.708528 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-598d67d848-dr22z" podStartSLOduration=2.418256482 podStartE2EDuration="6.708514156s" totalImagesPullingTime="3.224171058s" totalInitContainerRuntime="1.066086616s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:32:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:32:26.707078052 +0000 UTC m=+284.681160581" watchObservedRunningTime="2026-08-04 10:32:26.708514156 +0000 UTC m=+284.682596677" Aug 04 10:32:34.409619 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:34.409587 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:34.509890 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:34.509850 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:37.694112 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:37.694084 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-598d67d848-dr22z" Aug 04 10:32:39.701406 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:39.701374 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:39.797305 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:39.797248 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:40.197667 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:40.197635 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:40.412235 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:40.412201 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:40.513688 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:40.513590 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:40.612454 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:40.612414 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:40.703718 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:40.703687 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:40.707968 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:40.707941 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-59766cd755-pl4cj"] Aug 04 10:32:40.711596 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:40.711571 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59766cd755-pl4cj" Aug 04 10:32:40.713933 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:40.713913 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-xbfjb\"" Aug 04 10:32:40.714070 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:40.713915 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Aug 04 10:32:40.715505 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:40.715292 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Aug 04 10:32:40.715505 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:40.715350 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Aug 04 10:32:40.725932 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:40.725903 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59766cd755-pl4cj"] Aug 04 10:32:40.805355 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:40.805316 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b047bdce-f1e4-4ffe-92d3-ffaf4d225065-metrics-cert\") pod \"lws-controller-manager-59766cd755-pl4cj\" (UID: \"b047bdce-f1e4-4ffe-92d3-ffaf4d225065\") " pod="openshift-lws-operator/lws-controller-manager-59766cd755-pl4cj" Aug 04 10:32:40.805542 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:40.805384 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b047bdce-f1e4-4ffe-92d3-ffaf4d225065-cert\") pod \"lws-controller-manager-59766cd755-pl4cj\" (UID: \"b047bdce-f1e4-4ffe-92d3-ffaf4d225065\") " pod="openshift-lws-operator/lws-controller-manager-59766cd755-pl4cj" Aug 04 10:32:40.805542 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:40.805428 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktlds\" (UniqueName: \"kubernetes.io/projected/b047bdce-f1e4-4ffe-92d3-ffaf4d225065-kube-api-access-ktlds\") pod \"lws-controller-manager-59766cd755-pl4cj\" (UID: \"b047bdce-f1e4-4ffe-92d3-ffaf4d225065\") " pod="openshift-lws-operator/lws-controller-manager-59766cd755-pl4cj" Aug 04 10:32:40.805542 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:40.805455 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b047bdce-f1e4-4ffe-92d3-ffaf4d225065-manager-config\") pod \"lws-controller-manager-59766cd755-pl4cj\" (UID: \"b047bdce-f1e4-4ffe-92d3-ffaf4d225065\") " pod="openshift-lws-operator/lws-controller-manager-59766cd755-pl4cj" Aug 04 10:32:40.906193 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:40.906162 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b047bdce-f1e4-4ffe-92d3-ffaf4d225065-metrics-cert\") pod \"lws-controller-manager-59766cd755-pl4cj\" (UID: \"b047bdce-f1e4-4ffe-92d3-ffaf4d225065\") " pod="openshift-lws-operator/lws-controller-manager-59766cd755-pl4cj" Aug 04 10:32:40.906379 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:40.906218 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b047bdce-f1e4-4ffe-92d3-ffaf4d225065-cert\") pod \"lws-controller-manager-59766cd755-pl4cj\" (UID: \"b047bdce-f1e4-4ffe-92d3-ffaf4d225065\") " pod="openshift-lws-operator/lws-controller-manager-59766cd755-pl4cj" Aug 04 10:32:40.906379 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:40.906252 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktlds\" (UniqueName: \"kubernetes.io/projected/b047bdce-f1e4-4ffe-92d3-ffaf4d225065-kube-api-access-ktlds\") pod \"lws-controller-manager-59766cd755-pl4cj\" (UID: \"b047bdce-f1e4-4ffe-92d3-ffaf4d225065\") " pod="openshift-lws-operator/lws-controller-manager-59766cd755-pl4cj" Aug 04 10:32:40.906379 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:40.906275 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b047bdce-f1e4-4ffe-92d3-ffaf4d225065-manager-config\") pod \"lws-controller-manager-59766cd755-pl4cj\" (UID: \"b047bdce-f1e4-4ffe-92d3-ffaf4d225065\") " pod="openshift-lws-operator/lws-controller-manager-59766cd755-pl4cj" Aug 04 10:32:40.907206 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:40.907173 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b047bdce-f1e4-4ffe-92d3-ffaf4d225065-manager-config\") pod \"lws-controller-manager-59766cd755-pl4cj\" (UID: \"b047bdce-f1e4-4ffe-92d3-ffaf4d225065\") " pod="openshift-lws-operator/lws-controller-manager-59766cd755-pl4cj" Aug 04 10:32:40.908922 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:40.908895 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b047bdce-f1e4-4ffe-92d3-ffaf4d225065-metrics-cert\") pod \"lws-controller-manager-59766cd755-pl4cj\" (UID: \"b047bdce-f1e4-4ffe-92d3-ffaf4d225065\") " pod="openshift-lws-operator/lws-controller-manager-59766cd755-pl4cj" Aug 04 10:32:40.909266 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:40.909250 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b047bdce-f1e4-4ffe-92d3-ffaf4d225065-cert\") pod \"lws-controller-manager-59766cd755-pl4cj\" (UID: \"b047bdce-f1e4-4ffe-92d3-ffaf4d225065\") " pod="openshift-lws-operator/lws-controller-manager-59766cd755-pl4cj" Aug 04 10:32:40.923347 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:40.923321 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktlds\" (UniqueName: \"kubernetes.io/projected/b047bdce-f1e4-4ffe-92d3-ffaf4d225065-kube-api-access-ktlds\") pod \"lws-controller-manager-59766cd755-pl4cj\" (UID: \"b047bdce-f1e4-4ffe-92d3-ffaf4d225065\") " pod="openshift-lws-operator/lws-controller-manager-59766cd755-pl4cj" Aug 04 10:32:40.974850 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:40.974816 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:41.022951 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:41.022921 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59766cd755-pl4cj" Aug 04 10:32:41.173059 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:41.173025 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59766cd755-pl4cj"] Aug 04 10:32:41.175870 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:32:41.175836 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb047bdce_f1e4_4ffe_92d3_ffaf4d225065.slice/crio-d862315a4c0b8ac57e000644027c41fbc8d9a84d8a4ac66d84bbf956614fa8db WatchSource:0}: Error finding container d862315a4c0b8ac57e000644027c41fbc8d9a84d8a4ac66d84bbf956614fa8db: Status 404 returned error can't find the container with id d862315a4c0b8ac57e000644027c41fbc8d9a84d8a4ac66d84bbf956614fa8db Aug 04 10:32:41.624844 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:41.624809 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:41.723578 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:41.723543 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:41.748225 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:41.748201 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59766cd755-pl4cj" event={"ID":"b047bdce-f1e4-4ffe-92d3-ffaf4d225065","Type":"ContainerStarted","Data":"d862315a4c0b8ac57e000644027c41fbc8d9a84d8a4ac66d84bbf956614fa8db"} Aug 04 10:32:42.429606 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:42.429580 2535 kubelet.go:1648] "Image garbage collection succeeded" Aug 04 10:32:42.704332 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:42.704233 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:43.758293 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:43.758255 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59766cd755-pl4cj" event={"ID":"b047bdce-f1e4-4ffe-92d3-ffaf4d225065","Type":"ContainerStarted","Data":"ec28d566e7afe96b8ee131fe0430ef7c734519b82c69c806a38d1fda496c7591"} Aug 04 10:32:43.758748 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:43.758383 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-59766cd755-pl4cj" Aug 04 10:32:43.776148 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:43.776094 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-59766cd755-pl4cj" podStartSLOduration=1.9738051730000001 podStartE2EDuration="3.77607412s" totalImagesPullingTime="1.802268947s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:32:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:32:43.775713143 +0000 UTC m=+301.749795698" watchObservedRunningTime="2026-08-04 10:32:43.77607412 +0000 UTC m=+301.750156651" Aug 04 10:32:47.882969 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:47.882935 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv"] Aug 04 10:32:47.887338 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:47.887315 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv" Aug 04 10:32:47.891970 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:47.891949 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Aug 04 10:32:47.892277 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:47.892257 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Aug 04 10:32:47.892462 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:47.892440 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Aug 04 10:32:47.892723 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:47.892701 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Aug 04 10:32:47.892822 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:47.892758 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Aug 04 10:32:47.892822 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:47.892769 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-dq2q9\"" Aug 04 10:32:47.893062 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:47.893047 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Aug 04 10:32:47.902171 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:47.902143 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv"] Aug 04 10:32:47.966718 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:47.966669 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a1a1cd50-d008-4bf4-89bb-43afe394c465-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-6wrhv\" (UID: \"a1a1cd50-d008-4bf4-89bb-43afe394c465\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv" Aug 04 10:32:47.966872 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:47.966724 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf7sh\" (UniqueName: \"kubernetes.io/projected/a1a1cd50-d008-4bf4-89bb-43afe394c465-kube-api-access-qf7sh\") pod \"istiod-openshift-gateway-5677ccb49b-6wrhv\" (UID: \"a1a1cd50-d008-4bf4-89bb-43afe394c465\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv" Aug 04 10:32:47.966872 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:47.966823 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1a1cd50-d008-4bf4-89bb-43afe394c465-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-6wrhv\" (UID: \"a1a1cd50-d008-4bf4-89bb-43afe394c465\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv" Aug 04 10:32:47.966872 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:47.966853 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1a1cd50-d008-4bf4-89bb-43afe394c465-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-6wrhv\" (UID: \"a1a1cd50-d008-4bf4-89bb-43afe394c465\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv" Aug 04 10:32:47.966973 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:47.966895 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a1a1cd50-d008-4bf4-89bb-43afe394c465-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-6wrhv\" (UID: \"a1a1cd50-d008-4bf4-89bb-43afe394c465\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv" Aug 04 10:32:47.966973 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:47.966913 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a1a1cd50-d008-4bf4-89bb-43afe394c465-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-6wrhv\" (UID: \"a1a1cd50-d008-4bf4-89bb-43afe394c465\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv" Aug 04 10:32:47.967035 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:47.966970 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a1a1cd50-d008-4bf4-89bb-43afe394c465-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-6wrhv\" (UID: \"a1a1cd50-d008-4bf4-89bb-43afe394c465\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv" Aug 04 10:32:48.067962 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:48.067930 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a1a1cd50-d008-4bf4-89bb-43afe394c465-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-6wrhv\" (UID: \"a1a1cd50-d008-4bf4-89bb-43afe394c465\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv" Aug 04 10:32:48.067962 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:48.067965 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf7sh\" (UniqueName: \"kubernetes.io/projected/a1a1cd50-d008-4bf4-89bb-43afe394c465-kube-api-access-qf7sh\") pod \"istiod-openshift-gateway-5677ccb49b-6wrhv\" (UID: \"a1a1cd50-d008-4bf4-89bb-43afe394c465\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv" Aug 04 10:32:48.068163 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:48.067994 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1a1cd50-d008-4bf4-89bb-43afe394c465-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-6wrhv\" (UID: \"a1a1cd50-d008-4bf4-89bb-43afe394c465\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv" Aug 04 10:32:48.068238 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:48.068217 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1a1cd50-d008-4bf4-89bb-43afe394c465-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-6wrhv\" (UID: \"a1a1cd50-d008-4bf4-89bb-43afe394c465\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv" Aug 04 10:32:48.068298 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:48.068284 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a1a1cd50-d008-4bf4-89bb-43afe394c465-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-6wrhv\" (UID: \"a1a1cd50-d008-4bf4-89bb-43afe394c465\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv" Aug 04 10:32:48.068334 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:48.068314 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a1a1cd50-d008-4bf4-89bb-43afe394c465-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-6wrhv\" (UID: \"a1a1cd50-d008-4bf4-89bb-43afe394c465\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv" Aug 04 10:32:48.068380 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:48.068355 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a1a1cd50-d008-4bf4-89bb-43afe394c465-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-6wrhv\" (UID: \"a1a1cd50-d008-4bf4-89bb-43afe394c465\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv" Aug 04 10:32:48.068994 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:48.068963 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a1a1cd50-d008-4bf4-89bb-43afe394c465-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-6wrhv\" (UID: \"a1a1cd50-d008-4bf4-89bb-43afe394c465\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv" Aug 04 10:32:48.070359 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:48.070324 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1a1cd50-d008-4bf4-89bb-43afe394c465-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-6wrhv\" (UID: \"a1a1cd50-d008-4bf4-89bb-43afe394c465\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv" Aug 04 10:32:48.070503 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:48.070400 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a1a1cd50-d008-4bf4-89bb-43afe394c465-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-6wrhv\" (UID: \"a1a1cd50-d008-4bf4-89bb-43afe394c465\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv" Aug 04 10:32:48.070574 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:48.070502 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a1a1cd50-d008-4bf4-89bb-43afe394c465-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-6wrhv\" (UID: \"a1a1cd50-d008-4bf4-89bb-43afe394c465\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv" Aug 04 10:32:48.070632 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:48.070573 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a1a1cd50-d008-4bf4-89bb-43afe394c465-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-6wrhv\" (UID: \"a1a1cd50-d008-4bf4-89bb-43afe394c465\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv" Aug 04 10:32:48.076791 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:48.076766 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1a1cd50-d008-4bf4-89bb-43afe394c465-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-6wrhv\" (UID: \"a1a1cd50-d008-4bf4-89bb-43afe394c465\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv" Aug 04 10:32:48.076901 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:48.076883 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf7sh\" (UniqueName: \"kubernetes.io/projected/a1a1cd50-d008-4bf4-89bb-43afe394c465-kube-api-access-qf7sh\") pod \"istiod-openshift-gateway-5677ccb49b-6wrhv\" (UID: \"a1a1cd50-d008-4bf4-89bb-43afe394c465\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv" Aug 04 10:32:48.197444 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:48.197366 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv" Aug 04 10:32:48.345697 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:32:48.345670 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1a1cd50_d008_4bf4_89bb_43afe394c465.slice/crio-ff54eb4638f804570accc755022253277c45256e9dfabb920ec03283eab86734 WatchSource:0}: Error finding container ff54eb4638f804570accc755022253277c45256e9dfabb920ec03283eab86734: Status 404 returned error can't find the container with id ff54eb4638f804570accc755022253277c45256e9dfabb920ec03283eab86734 Aug 04 10:32:48.347682 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:48.347664 2535 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 10:32:48.357168 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:48.357145 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv"] Aug 04 10:32:48.783233 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:48.783198 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv" event={"ID":"a1a1cd50-d008-4bf4-89bb-43afe394c465","Type":"ContainerStarted","Data":"ff54eb4638f804570accc755022253277c45256e9dfabb920ec03283eab86734"} Aug 04 10:32:50.619550 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:50.619477 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:51.170241 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:51.170201 2535 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 10:32:51.170385 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:51.170280 2535 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 04 10:32:51.798022 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:51.797986 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv" event={"ID":"a1a1cd50-d008-4bf4-89bb-43afe394c465","Type":"ContainerStarted","Data":"3c19fe5cf41a4547ae88c68f917289aa9e1c3c908c949f8427f612f07f12e402"} Aug 04 10:32:51.798500 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:51.798113 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv" Aug 04 10:32:51.821643 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:51.821592 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv" podStartSLOduration=1.999447862 podStartE2EDuration="4.821572788s" totalImagesPullingTime="2.822124926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:32:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:32:51.82089438 +0000 UTC m=+309.794976925" watchObservedRunningTime="2026-08-04 10:32:51.821572788 +0000 UTC m=+309.795655317" Aug 04 10:32:51.832934 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:51.832909 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:51.935183 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:51.935153 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:52.027249 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:52.027219 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:52.142271 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:52.142145 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:52.227856 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:52.227777 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:52.542440 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:52.542411 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:52.749269 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:52.749237 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:52.804362 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:52.804295 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6wrhv" Aug 04 10:32:52.916944 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:52.916901 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:32:54.764509 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:32:54.764455 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-59766cd755-pl4cj" Aug 04 10:33:12.065696 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:33:12.065658 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:33:27.248512 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:33:27.248457 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:33:29.095216 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:33:29.095171 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:33:41.483937 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:33:41.483897 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:33:41.568844 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:33:41.568809 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:33:54.022759 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:33:54.022719 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:33:56.423342 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:33:56.423313 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:33:56.828107 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:33:56.828071 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:33:58.262032 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:33:58.261998 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:33:58.360973 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:33:58.360936 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:33:59.226182 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:33:59.226148 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:33:59.462474 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:33:59.462439 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:33:59.626335 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:33:59.626303 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:00.229910 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:00.229879 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:01.127177 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:01.127113 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:01.230535 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:01.230475 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:03.651227 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:03.651193 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:03.755004 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:03.754969 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:06.669548 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:06.669515 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:06.768078 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:06.768044 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:10.203153 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:10.203114 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:10.914021 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:10.913984 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:11.840409 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:11.840373 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:11.946799 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:11.946765 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:12.040730 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:12.040694 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:12.148938 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:12.148849 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:16.218087 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:16.218054 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:16.316961 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:16.316925 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:16.400614 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:16.400576 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:20.185993 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:20.185959 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:23.264263 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:23.264225 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:23.364348 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:23.364312 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:25.877270 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:25.877233 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:25.980156 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:25.980118 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:26.930787 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:26.930750 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:27.034376 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:27.034338 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:30.934385 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:30.934348 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:31.035176 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:31.035142 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:31.100637 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:31.100609 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:31.255946 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:31.255868 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:31.357247 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:31.357217 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:31.656419 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:31.656391 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:31.730416 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:31.730382 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:36.004058 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:36.004019 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:36.102468 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:36.102432 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:36.202331 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:36.202298 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:43.010857 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:43.010820 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:43.110543 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:43.110461 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:50.734861 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:50.734826 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:34:50.835122 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:34:50.835090 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:35:03.729877 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:03.729832 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:35:03.733569 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:03.733543 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-7m7f5"] Aug 04 10:35:03.736997 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:03.736981 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-7m7f5" Aug 04 10:35:03.739695 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:03.739676 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-r7drb\"" Aug 04 10:35:03.739804 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:03.739756 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Aug 04 10:35:03.740117 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:03.740099 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Aug 04 10:35:03.740166 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:03.740128 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Aug 04 10:35:03.754069 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:03.754049 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-7m7f5"] Aug 04 10:35:03.829541 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:03.829511 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-7m7f5"] Aug 04 10:35:03.895519 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:03.895455 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g57vp\" (UniqueName: \"kubernetes.io/projected/82a2f6d9-3fc3-4196-ae8c-1b517f48de1a-kube-api-access-g57vp\") pod \"limitador-limitador-859c4d96d6-7m7f5\" (UID: \"82a2f6d9-3fc3-4196-ae8c-1b517f48de1a\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-7m7f5" Aug 04 10:35:03.895689 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:03.895540 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/82a2f6d9-3fc3-4196-ae8c-1b517f48de1a-config-file\") pod \"limitador-limitador-859c4d96d6-7m7f5\" (UID: \"82a2f6d9-3fc3-4196-ae8c-1b517f48de1a\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-7m7f5" Aug 04 10:35:03.996554 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:03.996466 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g57vp\" (UniqueName: \"kubernetes.io/projected/82a2f6d9-3fc3-4196-ae8c-1b517f48de1a-kube-api-access-g57vp\") pod \"limitador-limitador-859c4d96d6-7m7f5\" (UID: \"82a2f6d9-3fc3-4196-ae8c-1b517f48de1a\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-7m7f5" Aug 04 10:35:03.996554 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:03.996525 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/82a2f6d9-3fc3-4196-ae8c-1b517f48de1a-config-file\") pod \"limitador-limitador-859c4d96d6-7m7f5\" (UID: \"82a2f6d9-3fc3-4196-ae8c-1b517f48de1a\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-7m7f5" Aug 04 10:35:03.997144 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:03.997119 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/82a2f6d9-3fc3-4196-ae8c-1b517f48de1a-config-file\") pod \"limitador-limitador-859c4d96d6-7m7f5\" (UID: \"82a2f6d9-3fc3-4196-ae8c-1b517f48de1a\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-7m7f5" Aug 04 10:35:04.006397 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:04.006376 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g57vp\" (UniqueName: \"kubernetes.io/projected/82a2f6d9-3fc3-4196-ae8c-1b517f48de1a-kube-api-access-g57vp\") pod \"limitador-limitador-859c4d96d6-7m7f5\" (UID: \"82a2f6d9-3fc3-4196-ae8c-1b517f48de1a\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-7m7f5" Aug 04 10:35:04.034766 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:04.034738 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:35:04.046968 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:04.046948 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-7m7f5" Aug 04 10:35:04.110377 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:04.110340 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:35:04.198550 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:04.198513 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-7m7f5"] Aug 04 10:35:04.200757 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:35:04.200729 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82a2f6d9_3fc3_4196_ae8c_1b517f48de1a.slice/crio-0522127528ee426059947c19bba953dc9ad74c4cbe700dd5899825aca625d55f WatchSource:0}: Error finding container 0522127528ee426059947c19bba953dc9ad74c4cbe700dd5899825aca625d55f: Status 404 returned error can't find the container with id 0522127528ee426059947c19bba953dc9ad74c4cbe700dd5899825aca625d55f Aug 04 10:35:04.343093 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:04.343065 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-7m7f5" event={"ID":"82a2f6d9-3fc3-4196-ae8c-1b517f48de1a","Type":"ContainerStarted","Data":"0522127528ee426059947c19bba953dc9ad74c4cbe700dd5899825aca625d55f"} Aug 04 10:35:04.647471 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:04.647355 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:35:04.747549 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:04.747511 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:35:07.356592 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:07.356561 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-7m7f5" event={"ID":"82a2f6d9-3fc3-4196-ae8c-1b517f48de1a","Type":"ContainerStarted","Data":"2603e5b1907c857ccacc8cdd09aa5e4d8005fc496960b81a165ee4ceba7397ee"} Aug 04 10:35:07.356971 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:07.356666 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-7m7f5" Aug 04 10:35:07.374402 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:07.374363 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-7m7f5" podStartSLOduration=1.6417662069999999 podStartE2EDuration="4.37435131s" totalImagesPullingTime="2.732585103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:35:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:35:07.374325167 +0000 UTC m=+445.348407710" watchObservedRunningTime="2026-08-04 10:35:07.37435131 +0000 UTC m=+445.348433834" Aug 04 10:35:17.989241 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:17.989206 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-7m7f5"] Aug 04 10:35:17.989732 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:17.989394 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-7m7f5" podUID="82a2f6d9-3fc3-4196-ae8c-1b517f48de1a" containerName="limitador" containerID="cri-o://2603e5b1907c857ccacc8cdd09aa5e4d8005fc496960b81a165ee4ceba7397ee" gracePeriod=30 Aug 04 10:35:17.990133 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:17.990113 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-7m7f5" Aug 04 10:35:18.404409 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:18.404377 2535 generic.go:356] "Generic (PLEG): container finished" podID="82a2f6d9-3fc3-4196-ae8c-1b517f48de1a" containerID="2603e5b1907c857ccacc8cdd09aa5e4d8005fc496960b81a165ee4ceba7397ee" exitCode=0 Aug 04 10:35:18.404575 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:18.404421 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-7m7f5" event={"ID":"82a2f6d9-3fc3-4196-ae8c-1b517f48de1a","Type":"ContainerDied","Data":"2603e5b1907c857ccacc8cdd09aa5e4d8005fc496960b81a165ee4ceba7397ee"} Aug 04 10:35:18.923282 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:18.923258 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-7m7f5" Aug 04 10:35:19.016429 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:19.016389 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g57vp\" (UniqueName: \"kubernetes.io/projected/82a2f6d9-3fc3-4196-ae8c-1b517f48de1a-kube-api-access-g57vp\") pod \"82a2f6d9-3fc3-4196-ae8c-1b517f48de1a\" (UID: \"82a2f6d9-3fc3-4196-ae8c-1b517f48de1a\") " Aug 04 10:35:19.016816 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:19.016475 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/82a2f6d9-3fc3-4196-ae8c-1b517f48de1a-config-file\") pod \"82a2f6d9-3fc3-4196-ae8c-1b517f48de1a\" (UID: \"82a2f6d9-3fc3-4196-ae8c-1b517f48de1a\") " Aug 04 10:35:19.016865 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:19.016813 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82a2f6d9-3fc3-4196-ae8c-1b517f48de1a-config-file" (OuterVolumeSpecName: "config-file") pod "82a2f6d9-3fc3-4196-ae8c-1b517f48de1a" (UID: "82a2f6d9-3fc3-4196-ae8c-1b517f48de1a"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:35:19.018519 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:19.018478 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82a2f6d9-3fc3-4196-ae8c-1b517f48de1a-kube-api-access-g57vp" (OuterVolumeSpecName: "kube-api-access-g57vp") pod "82a2f6d9-3fc3-4196-ae8c-1b517f48de1a" (UID: "82a2f6d9-3fc3-4196-ae8c-1b517f48de1a"). InnerVolumeSpecName "kube-api-access-g57vp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:35:19.117580 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:19.117513 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g57vp\" (UniqueName: \"kubernetes.io/projected/82a2f6d9-3fc3-4196-ae8c-1b517f48de1a-kube-api-access-g57vp\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:35:19.117580 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:19.117536 2535 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/82a2f6d9-3fc3-4196-ae8c-1b517f48de1a-config-file\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:35:19.409684 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:19.409601 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-7m7f5" event={"ID":"82a2f6d9-3fc3-4196-ae8c-1b517f48de1a","Type":"ContainerDied","Data":"0522127528ee426059947c19bba953dc9ad74c4cbe700dd5899825aca625d55f"} Aug 04 10:35:19.409684 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:19.409639 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-7m7f5" Aug 04 10:35:19.409684 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:19.409664 2535 scope.go:117] "RemoveContainer" containerID="2603e5b1907c857ccacc8cdd09aa5e4d8005fc496960b81a165ee4ceba7397ee" Aug 04 10:35:19.430160 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:19.430137 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-7m7f5"] Aug 04 10:35:19.435238 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:19.435218 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-7m7f5"] Aug 04 10:35:20.479159 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:20.479116 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82a2f6d9-3fc3-4196-ae8c-1b517f48de1a" path="/var/lib/kubelet/pods/82a2f6d9-3fc3-4196-ae8c-1b517f48de1a/volumes" Aug 04 10:35:23.502001 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:23.501973 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:35:23.605466 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:23.605434 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:35:23.970472 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:23.970443 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hzmvq"] Aug 04 10:35:23.970774 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:23.970761 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82a2f6d9-3fc3-4196-ae8c-1b517f48de1a" containerName="limitador" Aug 04 10:35:23.970823 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:23.970776 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="82a2f6d9-3fc3-4196-ae8c-1b517f48de1a" containerName="limitador" Aug 04 10:35:23.970867 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:23.970827 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="82a2f6d9-3fc3-4196-ae8c-1b517f48de1a" containerName="limitador" Aug 04 10:35:23.976824 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:23.976809 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hzmvq" Aug 04 10:35:23.978957 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:23.978935 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Aug 04 10:35:23.979095 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:23.978991 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 04 10:35:23.979095 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:23.978992 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Aug 04 10:35:23.979480 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:23.979465 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-5dvpt\"" Aug 04 10:35:23.982627 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:23.982607 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hzmvq"] Aug 04 10:35:23.999675 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:23.999642 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:35:24.160842 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:24.160805 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gld5f\" (UniqueName: \"kubernetes.io/projected/bc1e76a3-d78b-4d45-8140-891118fe0a51-kube-api-access-gld5f\") pod \"postgres-69fb4594cf-hzmvq\" (UID: \"bc1e76a3-d78b-4d45-8140-891118fe0a51\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hzmvq" Aug 04 10:35:24.160842 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:24.160848 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/bc1e76a3-d78b-4d45-8140-891118fe0a51-data\") pod \"postgres-69fb4594cf-hzmvq\" (UID: \"bc1e76a3-d78b-4d45-8140-891118fe0a51\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hzmvq" Aug 04 10:35:24.261322 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:24.261239 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gld5f\" (UniqueName: \"kubernetes.io/projected/bc1e76a3-d78b-4d45-8140-891118fe0a51-kube-api-access-gld5f\") pod \"postgres-69fb4594cf-hzmvq\" (UID: \"bc1e76a3-d78b-4d45-8140-891118fe0a51\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hzmvq" Aug 04 10:35:24.261322 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:24.261281 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/bc1e76a3-d78b-4d45-8140-891118fe0a51-data\") pod \"postgres-69fb4594cf-hzmvq\" (UID: \"bc1e76a3-d78b-4d45-8140-891118fe0a51\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hzmvq" Aug 04 10:35:24.261754 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:24.261735 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/bc1e76a3-d78b-4d45-8140-891118fe0a51-data\") pod \"postgres-69fb4594cf-hzmvq\" (UID: \"bc1e76a3-d78b-4d45-8140-891118fe0a51\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hzmvq" Aug 04 10:35:24.271983 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:24.271957 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gld5f\" (UniqueName: \"kubernetes.io/projected/bc1e76a3-d78b-4d45-8140-891118fe0a51-kube-api-access-gld5f\") pod \"postgres-69fb4594cf-hzmvq\" (UID: \"bc1e76a3-d78b-4d45-8140-891118fe0a51\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hzmvq" Aug 04 10:35:24.288781 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:24.288762 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hzmvq" Aug 04 10:35:24.416769 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:24.416738 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hzmvq"] Aug 04 10:35:24.419799 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:35:24.419774 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc1e76a3_d78b_4d45_8140_891118fe0a51.slice/crio-0ad041e71f60583a4ef7d6beb163890049f319eff47b5aa48b64adcc241d0c86 WatchSource:0}: Error finding container 0ad041e71f60583a4ef7d6beb163890049f319eff47b5aa48b64adcc241d0c86: Status 404 returned error can't find the container with id 0ad041e71f60583a4ef7d6beb163890049f319eff47b5aa48b64adcc241d0c86 Aug 04 10:35:24.432851 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:24.432823 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hzmvq" event={"ID":"bc1e76a3-d78b-4d45-8140-891118fe0a51","Type":"ContainerStarted","Data":"0ad041e71f60583a4ef7d6beb163890049f319eff47b5aa48b64adcc241d0c86"} Aug 04 10:35:30.462338 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:30.462298 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hzmvq" event={"ID":"bc1e76a3-d78b-4d45-8140-891118fe0a51","Type":"ContainerStarted","Data":"166827fa3755628affb9f1533730f7ea98e063ce9ffb7ceb9a944259706777ff"} Aug 04 10:35:30.462720 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:30.462380 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hzmvq" Aug 04 10:35:30.480613 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:30.480560 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hzmvq" podStartSLOduration=2.075043782 podStartE2EDuration="7.480541171s" totalImagesPullingTime="5.405497389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:35:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:35:30.479716881 +0000 UTC m=+468.453799410" watchObservedRunningTime="2026-08-04 10:35:30.480541171 +0000 UTC m=+468.454623699" Aug 04 10:35:36.494865 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:36.494835 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hzmvq" Aug 04 10:35:36.829393 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:36.829360 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:35:37.620805 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:37.620776 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:35:43.765138 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:43.765107 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:35:43.786675 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:43.786648 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-699747fc5-xq54h"] Aug 04 10:35:43.791169 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:43.791149 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-699747fc5-xq54h" Aug 04 10:35:43.793555 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:43.793527 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Aug 04 10:35:43.793555 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:43.793545 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Aug 04 10:35:43.793745 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:43.793602 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7tzq4\"" Aug 04 10:35:43.799905 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:43.799843 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-699747fc5-xq54h"] Aug 04 10:35:43.827331 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:43.827302 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5c2c3a64-5015-4de2-a904-bc91076cba23-webhook-cert\") pod \"maas-controller-699747fc5-xq54h\" (UID: \"5c2c3a64-5015-4de2-a904-bc91076cba23\") " pod="opendatahub/maas-controller-699747fc5-xq54h" Aug 04 10:35:43.827462 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:43.827376 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2k7l\" (UniqueName: \"kubernetes.io/projected/5c2c3a64-5015-4de2-a904-bc91076cba23-kube-api-access-m2k7l\") pod \"maas-controller-699747fc5-xq54h\" (UID: \"5c2c3a64-5015-4de2-a904-bc91076cba23\") " pod="opendatahub/maas-controller-699747fc5-xq54h" Aug 04 10:35:43.884194 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:43.884169 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:35:43.928688 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:43.928661 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5c2c3a64-5015-4de2-a904-bc91076cba23-webhook-cert\") pod \"maas-controller-699747fc5-xq54h\" (UID: \"5c2c3a64-5015-4de2-a904-bc91076cba23\") " pod="opendatahub/maas-controller-699747fc5-xq54h" Aug 04 10:35:43.928823 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:43.928710 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2k7l\" (UniqueName: \"kubernetes.io/projected/5c2c3a64-5015-4de2-a904-bc91076cba23-kube-api-access-m2k7l\") pod \"maas-controller-699747fc5-xq54h\" (UID: \"5c2c3a64-5015-4de2-a904-bc91076cba23\") " pod="opendatahub/maas-controller-699747fc5-xq54h" Aug 04 10:35:43.931024 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:43.931004 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5c2c3a64-5015-4de2-a904-bc91076cba23-webhook-cert\") pod \"maas-controller-699747fc5-xq54h\" (UID: \"5c2c3a64-5015-4de2-a904-bc91076cba23\") " pod="opendatahub/maas-controller-699747fc5-xq54h" Aug 04 10:35:43.944944 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:43.944924 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2k7l\" (UniqueName: \"kubernetes.io/projected/5c2c3a64-5015-4de2-a904-bc91076cba23-kube-api-access-m2k7l\") pod \"maas-controller-699747fc5-xq54h\" (UID: \"5c2c3a64-5015-4de2-a904-bc91076cba23\") " pod="opendatahub/maas-controller-699747fc5-xq54h" Aug 04 10:35:43.972804 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:43.972768 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:35:43.973405 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:43.973388 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67c8859cd8-8nx9w"] Aug 04 10:35:43.976673 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:43.976658 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67c8859cd8-8nx9w" Aug 04 10:35:43.989949 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:43.989929 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67c8859cd8-8nx9w"] Aug 04 10:35:44.030003 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:44.029980 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpgjb\" (UniqueName: \"kubernetes.io/projected/dda5dfce-9fdb-47c6-8d4b-faf00ec66182-kube-api-access-cpgjb\") pod \"maas-controller-67c8859cd8-8nx9w\" (UID: \"dda5dfce-9fdb-47c6-8d4b-faf00ec66182\") " pod="opendatahub/maas-controller-67c8859cd8-8nx9w" Aug 04 10:35:44.030099 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:44.030078 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dda5dfce-9fdb-47c6-8d4b-faf00ec66182-webhook-cert\") pod \"maas-controller-67c8859cd8-8nx9w\" (UID: \"dda5dfce-9fdb-47c6-8d4b-faf00ec66182\") " pod="opendatahub/maas-controller-67c8859cd8-8nx9w" Aug 04 10:35:44.104682 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:44.104654 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-699747fc5-xq54h" Aug 04 10:35:44.130776 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:44.130740 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dda5dfce-9fdb-47c6-8d4b-faf00ec66182-webhook-cert\") pod \"maas-controller-67c8859cd8-8nx9w\" (UID: \"dda5dfce-9fdb-47c6-8d4b-faf00ec66182\") " pod="opendatahub/maas-controller-67c8859cd8-8nx9w" Aug 04 10:35:44.130890 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:44.130830 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpgjb\" (UniqueName: \"kubernetes.io/projected/dda5dfce-9fdb-47c6-8d4b-faf00ec66182-kube-api-access-cpgjb\") pod \"maas-controller-67c8859cd8-8nx9w\" (UID: \"dda5dfce-9fdb-47c6-8d4b-faf00ec66182\") " pod="opendatahub/maas-controller-67c8859cd8-8nx9w" Aug 04 10:35:44.133086 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:44.133064 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dda5dfce-9fdb-47c6-8d4b-faf00ec66182-webhook-cert\") pod \"maas-controller-67c8859cd8-8nx9w\" (UID: \"dda5dfce-9fdb-47c6-8d4b-faf00ec66182\") " pod="opendatahub/maas-controller-67c8859cd8-8nx9w" Aug 04 10:35:44.138851 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:44.138827 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpgjb\" (UniqueName: \"kubernetes.io/projected/dda5dfce-9fdb-47c6-8d4b-faf00ec66182-kube-api-access-cpgjb\") pod \"maas-controller-67c8859cd8-8nx9w\" (UID: \"dda5dfce-9fdb-47c6-8d4b-faf00ec66182\") " pod="opendatahub/maas-controller-67c8859cd8-8nx9w" Aug 04 10:35:44.240053 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:44.240028 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-699747fc5-xq54h"] Aug 04 10:35:44.242141 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:35:44.242113 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c2c3a64_5015_4de2_a904_bc91076cba23.slice/crio-66046e299873037001d82ad444e2d0440f6bc9aca5beb38bc2c0446216009fbe WatchSource:0}: Error finding container 66046e299873037001d82ad444e2d0440f6bc9aca5beb38bc2c0446216009fbe: Status 404 returned error can't find the container with id 66046e299873037001d82ad444e2d0440f6bc9aca5beb38bc2c0446216009fbe Aug 04 10:35:44.287007 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:44.286934 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67c8859cd8-8nx9w" Aug 04 10:35:44.418032 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:44.418003 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67c8859cd8-8nx9w"] Aug 04 10:35:44.420877 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:35:44.420846 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddda5dfce_9fdb_47c6_8d4b_faf00ec66182.slice/crio-62b112d381bfd3ef9e6c30f80c97d4d67feac916fe4667db0f1312ac774da6a7 WatchSource:0}: Error finding container 62b112d381bfd3ef9e6c30f80c97d4d67feac916fe4667db0f1312ac774da6a7: Status 404 returned error can't find the container with id 62b112d381bfd3ef9e6c30f80c97d4d67feac916fe4667db0f1312ac774da6a7 Aug 04 10:35:44.517542 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:44.517505 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67c8859cd8-8nx9w" event={"ID":"dda5dfce-9fdb-47c6-8d4b-faf00ec66182","Type":"ContainerStarted","Data":"62b112d381bfd3ef9e6c30f80c97d4d67feac916fe4667db0f1312ac774da6a7"} Aug 04 10:35:44.518659 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:44.518627 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-699747fc5-xq54h" event={"ID":"5c2c3a64-5015-4de2-a904-bc91076cba23","Type":"ContainerStarted","Data":"66046e299873037001d82ad444e2d0440f6bc9aca5beb38bc2c0446216009fbe"} Aug 04 10:35:46.901291 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:46.901247 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:35:46.999549 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:46.999467 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:35:47.534331 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:47.534292 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-699747fc5-xq54h" event={"ID":"5c2c3a64-5015-4de2-a904-bc91076cba23","Type":"ContainerStarted","Data":"d507812fd81393208cd0d094f146cd46eb504805d65de2365b9ae2d75f753af3"} Aug 04 10:35:47.534534 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:47.534451 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-699747fc5-xq54h" Aug 04 10:35:47.535734 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:47.535706 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67c8859cd8-8nx9w" event={"ID":"dda5dfce-9fdb-47c6-8d4b-faf00ec66182","Type":"ContainerStarted","Data":"1361f6d7a0e2e977e8d0cc9846a6b88571ee778d03d30ea999187d363498bdd4"} Aug 04 10:35:47.535854 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:47.535837 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67c8859cd8-8nx9w" Aug 04 10:35:47.553046 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:47.553001 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-699747fc5-xq54h" podStartSLOduration=2.066937496 podStartE2EDuration="4.55298797s" totalImagesPullingTime="2.486050474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:35:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:35:47.550980726 +0000 UTC m=+485.525063264" watchObservedRunningTime="2026-08-04 10:35:47.55298797 +0000 UTC m=+485.527070496" Aug 04 10:35:47.570735 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:47.570625 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67c8859cd8-8nx9w" podStartSLOduration=2.25685416 podStartE2EDuration="4.570607836s" totalImagesPullingTime="2.313753676s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:35:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:35:47.569617265 +0000 UTC m=+485.543699793" watchObservedRunningTime="2026-08-04 10:35:47.570607836 +0000 UTC m=+485.544690365" Aug 04 10:35:58.542425 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:58.542396 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67c8859cd8-8nx9w" Aug 04 10:35:58.542951 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:58.542465 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-699747fc5-xq54h" Aug 04 10:35:58.611593 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:58.611558 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-699747fc5-xq54h"] Aug 04 10:35:58.611811 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:58.611750 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-699747fc5-xq54h" podUID="5c2c3a64-5015-4de2-a904-bc91076cba23" containerName="manager" containerID="cri-o://d507812fd81393208cd0d094f146cd46eb504805d65de2365b9ae2d75f753af3" gracePeriod=30 Aug 04 10:35:58.859693 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:58.859670 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-699747fc5-xq54h" Aug 04 10:35:58.961797 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:58.961766 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2k7l\" (UniqueName: \"kubernetes.io/projected/5c2c3a64-5015-4de2-a904-bc91076cba23-kube-api-access-m2k7l\") pod \"5c2c3a64-5015-4de2-a904-bc91076cba23\" (UID: \"5c2c3a64-5015-4de2-a904-bc91076cba23\") " Aug 04 10:35:58.961968 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:58.961820 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5c2c3a64-5015-4de2-a904-bc91076cba23-webhook-cert\") pod \"5c2c3a64-5015-4de2-a904-bc91076cba23\" (UID: \"5c2c3a64-5015-4de2-a904-bc91076cba23\") " Aug 04 10:35:58.963948 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:58.963918 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c2c3a64-5015-4de2-a904-bc91076cba23-kube-api-access-m2k7l" (OuterVolumeSpecName: "kube-api-access-m2k7l") pod "5c2c3a64-5015-4de2-a904-bc91076cba23" (UID: "5c2c3a64-5015-4de2-a904-bc91076cba23"). InnerVolumeSpecName "kube-api-access-m2k7l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:35:58.964048 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:58.963948 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c2c3a64-5015-4de2-a904-bc91076cba23-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5c2c3a64-5015-4de2-a904-bc91076cba23" (UID: "5c2c3a64-5015-4de2-a904-bc91076cba23"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 10:35:59.063269 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:59.063241 2535 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5c2c3a64-5015-4de2-a904-bc91076cba23-webhook-cert\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:35:59.063269 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:59.063268 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2k7l\" (UniqueName: \"kubernetes.io/projected/5c2c3a64-5015-4de2-a904-bc91076cba23-kube-api-access-m2k7l\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:35:59.586655 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:59.586620 2535 generic.go:356] "Generic (PLEG): container finished" podID="5c2c3a64-5015-4de2-a904-bc91076cba23" containerID="d507812fd81393208cd0d094f146cd46eb504805d65de2365b9ae2d75f753af3" exitCode=0 Aug 04 10:35:59.586655 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:59.586656 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-699747fc5-xq54h" event={"ID":"5c2c3a64-5015-4de2-a904-bc91076cba23","Type":"ContainerDied","Data":"d507812fd81393208cd0d094f146cd46eb504805d65de2365b9ae2d75f753af3"} Aug 04 10:35:59.587102 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:59.586667 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-699747fc5-xq54h" Aug 04 10:35:59.587102 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:59.586692 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-699747fc5-xq54h" event={"ID":"5c2c3a64-5015-4de2-a904-bc91076cba23","Type":"ContainerDied","Data":"66046e299873037001d82ad444e2d0440f6bc9aca5beb38bc2c0446216009fbe"} Aug 04 10:35:59.587102 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:59.586725 2535 scope.go:117] "RemoveContainer" containerID="d507812fd81393208cd0d094f146cd46eb504805d65de2365b9ae2d75f753af3" Aug 04 10:35:59.595441 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:59.595409 2535 scope.go:117] "RemoveContainer" containerID="d507812fd81393208cd0d094f146cd46eb504805d65de2365b9ae2d75f753af3" Aug 04 10:35:59.595685 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:35:59.595667 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d507812fd81393208cd0d094f146cd46eb504805d65de2365b9ae2d75f753af3\": container with ID starting with d507812fd81393208cd0d094f146cd46eb504805d65de2365b9ae2d75f753af3 not found: ID does not exist" containerID="d507812fd81393208cd0d094f146cd46eb504805d65de2365b9ae2d75f753af3" Aug 04 10:35:59.595736 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:59.595692 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d507812fd81393208cd0d094f146cd46eb504805d65de2365b9ae2d75f753af3"} err="failed to get container status \"d507812fd81393208cd0d094f146cd46eb504805d65de2365b9ae2d75f753af3\": rpc error: code = NotFound desc = could not find container \"d507812fd81393208cd0d094f146cd46eb504805d65de2365b9ae2d75f753af3\": container with ID starting with d507812fd81393208cd0d094f146cd46eb504805d65de2365b9ae2d75f753af3 not found: ID does not exist" Aug 04 10:35:59.607803 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:59.607781 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-699747fc5-xq54h"] Aug 04 10:35:59.612316 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:35:59.612295 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-699747fc5-xq54h"] Aug 04 10:36:00.478932 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:00.478899 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c2c3a64-5015-4de2-a904-bc91076cba23" path="/var/lib/kubelet/pods/5c2c3a64-5015-4de2-a904-bc91076cba23/volumes" Aug 04 10:36:17.718245 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:17.718209 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:36:17.803101 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:17.803066 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:36:19.269663 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:19.269625 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:36:19.374277 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:19.374234 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:36:19.867235 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:19.867203 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:36:20.001114 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.001084 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-whd9j"] Aug 04 10:36:20.001416 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.001404 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c2c3a64-5015-4de2-a904-bc91076cba23" containerName="manager" Aug 04 10:36:20.001459 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.001418 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c2c3a64-5015-4de2-a904-bc91076cba23" containerName="manager" Aug 04 10:36:20.001520 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.001467 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c2c3a64-5015-4de2-a904-bc91076cba23" containerName="manager" Aug 04 10:36:20.003396 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.003374 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-whd9j" Aug 04 10:36:20.006446 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.006423 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-6dlr7\"" Aug 04 10:36:20.006965 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.006950 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Aug 04 10:36:20.023379 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.023351 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-whd9j"] Aug 04 10:36:20.027452 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.027414 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2d7c7cf-b757-4023-a277-38570756f493-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-whd9j\" (UID: \"e2d7c7cf-b757-4023-a277-38570756f493\") " pod="openshift-ingress/payload-processing-5cc6cd444b-whd9j" Aug 04 10:36:20.027574 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.027541 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5j2x\" (UniqueName: \"kubernetes.io/projected/e2d7c7cf-b757-4023-a277-38570756f493-kube-api-access-q5j2x\") pod \"payload-processing-5cc6cd444b-whd9j\" (UID: \"e2d7c7cf-b757-4023-a277-38570756f493\") " pod="openshift-ingress/payload-processing-5cc6cd444b-whd9j" Aug 04 10:36:20.051915 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.051889 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-zxvgh"] Aug 04 10:36:20.054708 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.054691 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-zxvgh" Aug 04 10:36:20.071503 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.066383 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-zxvgh"] Aug 04 10:36:20.128962 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.128884 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5j2x\" (UniqueName: \"kubernetes.io/projected/e2d7c7cf-b757-4023-a277-38570756f493-kube-api-access-q5j2x\") pod \"payload-processing-5cc6cd444b-whd9j\" (UID: \"e2d7c7cf-b757-4023-a277-38570756f493\") " pod="openshift-ingress/payload-processing-5cc6cd444b-whd9j" Aug 04 10:36:20.128962 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.128946 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j7lg\" (UniqueName: \"kubernetes.io/projected/c71d8203-4632-4601-a247-8d255666d0d5-kube-api-access-7j7lg\") pod \"payload-pre-processing-687dbb6cf6-zxvgh\" (UID: \"c71d8203-4632-4601-a247-8d255666d0d5\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-zxvgh" Aug 04 10:36:20.129171 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.128968 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2d7c7cf-b757-4023-a277-38570756f493-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-whd9j\" (UID: \"e2d7c7cf-b757-4023-a277-38570756f493\") " pod="openshift-ingress/payload-processing-5cc6cd444b-whd9j" Aug 04 10:36:20.129171 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.128987 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c71d8203-4632-4601-a247-8d255666d0d5-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-zxvgh\" (UID: \"c71d8203-4632-4601-a247-8d255666d0d5\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-zxvgh" Aug 04 10:36:20.129643 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.129623 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2d7c7cf-b757-4023-a277-38570756f493-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-whd9j\" (UID: \"e2d7c7cf-b757-4023-a277-38570756f493\") " pod="openshift-ingress/payload-processing-5cc6cd444b-whd9j" Aug 04 10:36:20.138920 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.138898 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5j2x\" (UniqueName: \"kubernetes.io/projected/e2d7c7cf-b757-4023-a277-38570756f493-kube-api-access-q5j2x\") pod \"payload-processing-5cc6cd444b-whd9j\" (UID: \"e2d7c7cf-b757-4023-a277-38570756f493\") " pod="openshift-ingress/payload-processing-5cc6cd444b-whd9j" Aug 04 10:36:20.230149 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.230111 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j7lg\" (UniqueName: \"kubernetes.io/projected/c71d8203-4632-4601-a247-8d255666d0d5-kube-api-access-7j7lg\") pod \"payload-pre-processing-687dbb6cf6-zxvgh\" (UID: \"c71d8203-4632-4601-a247-8d255666d0d5\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-zxvgh" Aug 04 10:36:20.230319 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.230167 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c71d8203-4632-4601-a247-8d255666d0d5-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-zxvgh\" (UID: \"c71d8203-4632-4601-a247-8d255666d0d5\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-zxvgh" Aug 04 10:36:20.230883 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.230864 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c71d8203-4632-4601-a247-8d255666d0d5-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-zxvgh\" (UID: \"c71d8203-4632-4601-a247-8d255666d0d5\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-zxvgh" Aug 04 10:36:20.238647 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.238623 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j7lg\" (UniqueName: \"kubernetes.io/projected/c71d8203-4632-4601-a247-8d255666d0d5-kube-api-access-7j7lg\") pod \"payload-pre-processing-687dbb6cf6-zxvgh\" (UID: \"c71d8203-4632-4601-a247-8d255666d0d5\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-zxvgh" Aug 04 10:36:20.313846 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.313818 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-whd9j" Aug 04 10:36:20.364844 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.364545 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-zxvgh" Aug 04 10:36:20.459292 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.459262 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-whd9j"] Aug 04 10:36:20.463261 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:36:20.463018 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2d7c7cf_b757_4023_a277_38570756f493.slice/crio-950786b8e10273fd284e610ae15f021a841ca4100e01e65b085962fbf09c8c09 WatchSource:0}: Error finding container 950786b8e10273fd284e610ae15f021a841ca4100e01e65b085962fbf09c8c09: Status 404 returned error can't find the container with id 950786b8e10273fd284e610ae15f021a841ca4100e01e65b085962fbf09c8c09 Aug 04 10:36:20.512013 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.511982 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-zxvgh"] Aug 04 10:36:20.514752 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:36:20.514725 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc71d8203_4632_4601_a247_8d255666d0d5.slice/crio-be9c41c4e0821cf99947cd93bfd5ab80f626b42049b892abfa53ea29b0a0232a WatchSource:0}: Error finding container be9c41c4e0821cf99947cd93bfd5ab80f626b42049b892abfa53ea29b0a0232a: Status 404 returned error can't find the container with id be9c41c4e0821cf99947cd93bfd5ab80f626b42049b892abfa53ea29b0a0232a Aug 04 10:36:20.674191 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.674106 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-zxvgh" event={"ID":"c71d8203-4632-4601-a247-8d255666d0d5","Type":"ContainerStarted","Data":"be9c41c4e0821cf99947cd93bfd5ab80f626b42049b892abfa53ea29b0a0232a"} Aug 04 10:36:20.675350 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.675324 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-whd9j" event={"ID":"e2d7c7cf-b757-4023-a277-38570756f493","Type":"ContainerStarted","Data":"950786b8e10273fd284e610ae15f021a841ca4100e01e65b085962fbf09c8c09"} Aug 04 10:36:20.963645 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:20.963562 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:36:23.692133 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:23.692100 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-zxvgh" event={"ID":"c71d8203-4632-4601-a247-8d255666d0d5","Type":"ContainerStarted","Data":"16f511c0e4f14016d354a23d750e12555c376a74819e246f8aa7ece38f596310"} Aug 04 10:36:23.692593 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:23.692222 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-zxvgh" Aug 04 10:36:23.693564 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:23.693539 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-whd9j" event={"ID":"e2d7c7cf-b757-4023-a277-38570756f493","Type":"ContainerStarted","Data":"1aa2d2b25c4e8a9e0354fa18d9937066e3f80d22f51d73f60c21e1290c29ce61"} Aug 04 10:36:23.693667 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:23.693651 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-whd9j" Aug 04 10:36:23.712661 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:23.712601 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-zxvgh" podStartSLOduration=1.021580755 podStartE2EDuration="3.712585663s" totalImagesPullingTime="2.691004908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:36:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:36:23.709864569 +0000 UTC m=+521.683947118" watchObservedRunningTime="2026-08-04 10:36:23.712585663 +0000 UTC m=+521.686668190" Aug 04 10:36:23.729470 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:23.729424 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-whd9j" podStartSLOduration=1.992319065 podStartE2EDuration="4.72941129s" totalImagesPullingTime="2.737092225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:36:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:36:23.72844157 +0000 UTC m=+521.702524122" watchObservedRunningTime="2026-08-04 10:36:23.72941129 +0000 UTC m=+521.703493816" Aug 04 10:36:31.402158 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:31.402113 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:36:31.500084 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:31.500053 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:36:34.699264 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:34.699191 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-zxvgh" Aug 04 10:36:34.699670 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:34.699271 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-whd9j" Aug 04 10:36:41.834603 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:41.834567 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw"] Aug 04 10:36:41.838173 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:41.838150 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw" Aug 04 10:36:41.840727 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:41.840710 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Aug 04 10:36:41.841536 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:41.841514 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Aug 04 10:36:41.841653 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:41.841520 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-ps8jd\"" Aug 04 10:36:41.841714 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:41.841518 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Aug 04 10:36:41.851296 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:41.851274 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw"] Aug 04 10:36:41.868623 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:41.868598 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:36:41.918802 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:41.918769 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stvmf\" (UniqueName: \"kubernetes.io/projected/776675a2-a82b-454d-a0d1-ea8140d2789b-kube-api-access-stvmf\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw\" (UID: \"776675a2-a82b-454d-a0d1-ea8140d2789b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw" Aug 04 10:36:41.918971 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:41.918872 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/776675a2-a82b-454d-a0d1-ea8140d2789b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw\" (UID: \"776675a2-a82b-454d-a0d1-ea8140d2789b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw" Aug 04 10:36:41.918971 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:41.918895 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/776675a2-a82b-454d-a0d1-ea8140d2789b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw\" (UID: \"776675a2-a82b-454d-a0d1-ea8140d2789b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw" Aug 04 10:36:41.918971 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:41.918910 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/776675a2-a82b-454d-a0d1-ea8140d2789b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw\" (UID: \"776675a2-a82b-454d-a0d1-ea8140d2789b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw" Aug 04 10:36:41.919120 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:41.918982 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/776675a2-a82b-454d-a0d1-ea8140d2789b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw\" (UID: \"776675a2-a82b-454d-a0d1-ea8140d2789b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw" Aug 04 10:36:41.919120 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:41.919051 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/776675a2-a82b-454d-a0d1-ea8140d2789b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw\" (UID: \"776675a2-a82b-454d-a0d1-ea8140d2789b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw" Aug 04 10:36:42.020104 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:42.020071 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stvmf\" (UniqueName: \"kubernetes.io/projected/776675a2-a82b-454d-a0d1-ea8140d2789b-kube-api-access-stvmf\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw\" (UID: \"776675a2-a82b-454d-a0d1-ea8140d2789b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw" Aug 04 10:36:42.020281 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:42.020172 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/776675a2-a82b-454d-a0d1-ea8140d2789b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw\" (UID: \"776675a2-a82b-454d-a0d1-ea8140d2789b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw" Aug 04 10:36:42.020281 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:42.020202 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/776675a2-a82b-454d-a0d1-ea8140d2789b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw\" (UID: \"776675a2-a82b-454d-a0d1-ea8140d2789b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw" Aug 04 10:36:42.020281 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:42.020223 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/776675a2-a82b-454d-a0d1-ea8140d2789b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw\" (UID: \"776675a2-a82b-454d-a0d1-ea8140d2789b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw" Aug 04 10:36:42.020281 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:42.020262 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/776675a2-a82b-454d-a0d1-ea8140d2789b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw\" (UID: \"776675a2-a82b-454d-a0d1-ea8140d2789b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw" Aug 04 10:36:42.020526 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:42.020377 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/776675a2-a82b-454d-a0d1-ea8140d2789b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw\" (UID: \"776675a2-a82b-454d-a0d1-ea8140d2789b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw" Aug 04 10:36:42.020686 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:42.020660 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/776675a2-a82b-454d-a0d1-ea8140d2789b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw\" (UID: \"776675a2-a82b-454d-a0d1-ea8140d2789b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw" Aug 04 10:36:42.020686 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:42.020673 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/776675a2-a82b-454d-a0d1-ea8140d2789b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw\" (UID: \"776675a2-a82b-454d-a0d1-ea8140d2789b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw" Aug 04 10:36:42.020826 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:42.020726 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/776675a2-a82b-454d-a0d1-ea8140d2789b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw\" (UID: \"776675a2-a82b-454d-a0d1-ea8140d2789b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw" Aug 04 10:36:42.022477 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:42.022454 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/776675a2-a82b-454d-a0d1-ea8140d2789b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw\" (UID: \"776675a2-a82b-454d-a0d1-ea8140d2789b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw" Aug 04 10:36:42.022641 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:42.022614 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/776675a2-a82b-454d-a0d1-ea8140d2789b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw\" (UID: \"776675a2-a82b-454d-a0d1-ea8140d2789b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw" Aug 04 10:36:42.028548 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:42.028519 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stvmf\" (UniqueName: \"kubernetes.io/projected/776675a2-a82b-454d-a0d1-ea8140d2789b-kube-api-access-stvmf\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw\" (UID: \"776675a2-a82b-454d-a0d1-ea8140d2789b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw" Aug 04 10:36:42.151675 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:42.151598 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw" Aug 04 10:36:42.285029 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:42.284884 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw"] Aug 04 10:36:42.287235 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:36:42.287207 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod776675a2_a82b_454d_a0d1_ea8140d2789b.slice/crio-9f0c9265800cbed6d67596a4e8d7780b1951d985854c951bfd996f19cf74ce5b WatchSource:0}: Error finding container 9f0c9265800cbed6d67596a4e8d7780b1951d985854c951bfd996f19cf74ce5b: Status 404 returned error can't find the container with id 9f0c9265800cbed6d67596a4e8d7780b1951d985854c951bfd996f19cf74ce5b Aug 04 10:36:42.776467 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:42.776424 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw" event={"ID":"776675a2-a82b-454d-a0d1-ea8140d2789b","Type":"ContainerStarted","Data":"9f0c9265800cbed6d67596a4e8d7780b1951d985854c951bfd996f19cf74ce5b"} Aug 04 10:36:43.782404 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:43.782366 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw" event={"ID":"776675a2-a82b-454d-a0d1-ea8140d2789b","Type":"ContainerStarted","Data":"11f1a1b7067aa4a65cadb90a1a4d2ecf844e220751d71be5cdf71ce11cce6d3b"} Aug 04 10:36:43.801133 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:43.801082 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw" podStartSLOduration=1.7646203599999999 podStartE2EDuration="2.801067613s" totalImagesPullingTime="1.036447253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:36:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:36:43.799259356 +0000 UTC m=+541.773341883" watchObservedRunningTime="2026-08-04 10:36:43.801067613 +0000 UTC m=+541.775150139" Aug 04 10:36:46.333870 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:46.333829 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv"] Aug 04 10:36:46.337392 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:46.337371 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv" Aug 04 10:36:46.339335 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:46.339314 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Aug 04 10:36:46.346296 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:46.346263 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv"] Aug 04 10:36:46.433884 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:46.433846 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:36:46.462743 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:46.462717 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f66196c5-ae6e-4922-aa61-61436f504b4e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mq8lv\" (UID: \"f66196c5-ae6e-4922-aa61-61436f504b4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv" Aug 04 10:36:46.462908 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:46.462767 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f66196c5-ae6e-4922-aa61-61436f504b4e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mq8lv\" (UID: \"f66196c5-ae6e-4922-aa61-61436f504b4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv" Aug 04 10:36:46.462908 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:46.462796 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f66196c5-ae6e-4922-aa61-61436f504b4e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mq8lv\" (UID: \"f66196c5-ae6e-4922-aa61-61436f504b4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv" Aug 04 10:36:46.462908 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:46.462864 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f66196c5-ae6e-4922-aa61-61436f504b4e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mq8lv\" (UID: \"f66196c5-ae6e-4922-aa61-61436f504b4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv" Aug 04 10:36:46.463057 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:46.462922 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f66196c5-ae6e-4922-aa61-61436f504b4e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mq8lv\" (UID: \"f66196c5-ae6e-4922-aa61-61436f504b4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv" Aug 04 10:36:46.463057 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:46.462953 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtfsd\" (UniqueName: \"kubernetes.io/projected/f66196c5-ae6e-4922-aa61-61436f504b4e-kube-api-access-dtfsd\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mq8lv\" (UID: \"f66196c5-ae6e-4922-aa61-61436f504b4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv" Aug 04 10:36:46.563975 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:46.563940 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f66196c5-ae6e-4922-aa61-61436f504b4e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mq8lv\" (UID: \"f66196c5-ae6e-4922-aa61-61436f504b4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv" Aug 04 10:36:46.564132 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:46.564007 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f66196c5-ae6e-4922-aa61-61436f504b4e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mq8lv\" (UID: \"f66196c5-ae6e-4922-aa61-61436f504b4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv" Aug 04 10:36:46.564132 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:46.564039 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtfsd\" (UniqueName: \"kubernetes.io/projected/f66196c5-ae6e-4922-aa61-61436f504b4e-kube-api-access-dtfsd\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mq8lv\" (UID: \"f66196c5-ae6e-4922-aa61-61436f504b4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv" Aug 04 10:36:46.564236 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:46.564176 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f66196c5-ae6e-4922-aa61-61436f504b4e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mq8lv\" (UID: \"f66196c5-ae6e-4922-aa61-61436f504b4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv" Aug 04 10:36:46.564292 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:46.564235 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f66196c5-ae6e-4922-aa61-61436f504b4e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mq8lv\" (UID: \"f66196c5-ae6e-4922-aa61-61436f504b4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv" Aug 04 10:36:46.564292 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:46.564263 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f66196c5-ae6e-4922-aa61-61436f504b4e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mq8lv\" (UID: \"f66196c5-ae6e-4922-aa61-61436f504b4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv" Aug 04 10:36:46.564467 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:46.564424 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f66196c5-ae6e-4922-aa61-61436f504b4e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mq8lv\" (UID: \"f66196c5-ae6e-4922-aa61-61436f504b4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv" Aug 04 10:36:46.564619 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:46.564568 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f66196c5-ae6e-4922-aa61-61436f504b4e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mq8lv\" (UID: \"f66196c5-ae6e-4922-aa61-61436f504b4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv" Aug 04 10:36:46.564695 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:46.564627 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f66196c5-ae6e-4922-aa61-61436f504b4e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mq8lv\" (UID: \"f66196c5-ae6e-4922-aa61-61436f504b4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv" Aug 04 10:36:46.566337 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:46.566314 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f66196c5-ae6e-4922-aa61-61436f504b4e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mq8lv\" (UID: \"f66196c5-ae6e-4922-aa61-61436f504b4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv" Aug 04 10:36:46.566667 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:46.566651 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f66196c5-ae6e-4922-aa61-61436f504b4e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mq8lv\" (UID: \"f66196c5-ae6e-4922-aa61-61436f504b4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv" Aug 04 10:36:46.573368 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:46.573337 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtfsd\" (UniqueName: \"kubernetes.io/projected/f66196c5-ae6e-4922-aa61-61436f504b4e-kube-api-access-dtfsd\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mq8lv\" (UID: \"f66196c5-ae6e-4922-aa61-61436f504b4e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv" Aug 04 10:36:46.648243 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:46.648183 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv" Aug 04 10:36:46.776629 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:46.776604 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv"] Aug 04 10:36:46.778858 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:36:46.778832 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf66196c5_ae6e_4922_aa61_61436f504b4e.slice/crio-ae7bbcecfb1ba1d7b6efd104a3d42e10244a00308b3eb04b5d6ea75f14db91ad WatchSource:0}: Error finding container ae7bbcecfb1ba1d7b6efd104a3d42e10244a00308b3eb04b5d6ea75f14db91ad: Status 404 returned error can't find the container with id ae7bbcecfb1ba1d7b6efd104a3d42e10244a00308b3eb04b5d6ea75f14db91ad Aug 04 10:36:46.798711 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:46.798684 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv" event={"ID":"f66196c5-ae6e-4922-aa61-61436f504b4e","Type":"ContainerStarted","Data":"ae7bbcecfb1ba1d7b6efd104a3d42e10244a00308b3eb04b5d6ea75f14db91ad"} Aug 04 10:36:47.804163 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:47.804113 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv" event={"ID":"f66196c5-ae6e-4922-aa61-61436f504b4e","Type":"ContainerStarted","Data":"a76c71d35b936343da47b46d5c909559795ab0a1a3041d5ab3c0576268862de5"} Aug 04 10:36:47.823137 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:47.823085 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv" podStartSLOduration=1.649708458 podStartE2EDuration="1.823069136s" totalImagesPullingTime="173.360678ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:36:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:36:47.822468051 +0000 UTC m=+545.796550589" watchObservedRunningTime="2026-08-04 10:36:47.823069136 +0000 UTC m=+545.797151665" Aug 04 10:36:52.151823 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:52.151786 2535 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw" Aug 04 10:36:52.152231 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:52.151893 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw" Aug 04 10:36:52.164425 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:52.164399 2535 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw" Aug 04 10:36:52.838956 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:52.838924 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw" Aug 04 10:36:56.648627 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:56.648584 2535 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv" Aug 04 10:36:56.649096 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:56.648656 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv" Aug 04 10:36:56.661331 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:56.661309 2535 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv" Aug 04 10:36:56.855979 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:36:56.855948 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mq8lv" Aug 04 10:37:03.914161 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:03.914119 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:37:08.172035 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:08.172003 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:37:11.831989 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:11.831957 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2"] Aug 04 10:37:11.835357 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:11.835337 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2" Aug 04 10:37:11.837416 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:11.837397 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Aug 04 10:37:11.844881 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:11.844853 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2"] Aug 04 10:37:11.867639 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:11.867617 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:37:11.894191 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:11.894163 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54995a67-600f-4bda-bbd0-c86307e0df08-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2\" (UID: \"54995a67-600f-4bda-bbd0-c86307e0df08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2" Aug 04 10:37:11.894320 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:11.894214 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54995a67-600f-4bda-bbd0-c86307e0df08-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2\" (UID: \"54995a67-600f-4bda-bbd0-c86307e0df08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2" Aug 04 10:37:11.894320 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:11.894284 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54995a67-600f-4bda-bbd0-c86307e0df08-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2\" (UID: \"54995a67-600f-4bda-bbd0-c86307e0df08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2" Aug 04 10:37:11.894444 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:11.894412 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54995a67-600f-4bda-bbd0-c86307e0df08-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2\" (UID: \"54995a67-600f-4bda-bbd0-c86307e0df08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2" Aug 04 10:37:11.894552 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:11.894534 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8t27\" (UniqueName: \"kubernetes.io/projected/54995a67-600f-4bda-bbd0-c86307e0df08-kube-api-access-z8t27\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2\" (UID: \"54995a67-600f-4bda-bbd0-c86307e0df08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2" Aug 04 10:37:11.894625 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:11.894569 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54995a67-600f-4bda-bbd0-c86307e0df08-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2\" (UID: \"54995a67-600f-4bda-bbd0-c86307e0df08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2" Aug 04 10:37:11.995673 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:11.995641 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54995a67-600f-4bda-bbd0-c86307e0df08-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2\" (UID: \"54995a67-600f-4bda-bbd0-c86307e0df08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2" Aug 04 10:37:11.995673 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:11.995682 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54995a67-600f-4bda-bbd0-c86307e0df08-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2\" (UID: \"54995a67-600f-4bda-bbd0-c86307e0df08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2" Aug 04 10:37:11.995975 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:11.995717 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8t27\" (UniqueName: \"kubernetes.io/projected/54995a67-600f-4bda-bbd0-c86307e0df08-kube-api-access-z8t27\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2\" (UID: \"54995a67-600f-4bda-bbd0-c86307e0df08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2" Aug 04 10:37:11.995975 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:11.995738 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54995a67-600f-4bda-bbd0-c86307e0df08-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2\" (UID: \"54995a67-600f-4bda-bbd0-c86307e0df08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2" Aug 04 10:37:11.995975 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:11.995762 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54995a67-600f-4bda-bbd0-c86307e0df08-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2\" (UID: \"54995a67-600f-4bda-bbd0-c86307e0df08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2" Aug 04 10:37:11.995975 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:11.995781 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54995a67-600f-4bda-bbd0-c86307e0df08-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2\" (UID: \"54995a67-600f-4bda-bbd0-c86307e0df08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2" Aug 04 10:37:11.996178 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:11.996114 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/54995a67-600f-4bda-bbd0-c86307e0df08-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2\" (UID: \"54995a67-600f-4bda-bbd0-c86307e0df08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2" Aug 04 10:37:11.996178 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:11.996145 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54995a67-600f-4bda-bbd0-c86307e0df08-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2\" (UID: \"54995a67-600f-4bda-bbd0-c86307e0df08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2" Aug 04 10:37:11.996178 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:11.996167 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/54995a67-600f-4bda-bbd0-c86307e0df08-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2\" (UID: \"54995a67-600f-4bda-bbd0-c86307e0df08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2" Aug 04 10:37:11.997985 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:11.997962 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/54995a67-600f-4bda-bbd0-c86307e0df08-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2\" (UID: \"54995a67-600f-4bda-bbd0-c86307e0df08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2" Aug 04 10:37:11.998429 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:11.998407 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54995a67-600f-4bda-bbd0-c86307e0df08-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2\" (UID: \"54995a67-600f-4bda-bbd0-c86307e0df08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2" Aug 04 10:37:12.003387 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:12.003361 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8t27\" (UniqueName: \"kubernetes.io/projected/54995a67-600f-4bda-bbd0-c86307e0df08-kube-api-access-z8t27\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2\" (UID: \"54995a67-600f-4bda-bbd0-c86307e0df08\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2" Aug 04 10:37:12.147133 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:12.147046 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2" Aug 04 10:37:12.275704 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:12.275675 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2"] Aug 04 10:37:12.278096 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:37:12.278068 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54995a67_600f_4bda_bbd0_c86307e0df08.slice/crio-7106e35d16b979cda061e0a0f6f5b85ad97d15e06bb2b6c33b0f8f50cf5ccc16 WatchSource:0}: Error finding container 7106e35d16b979cda061e0a0f6f5b85ad97d15e06bb2b6c33b0f8f50cf5ccc16: Status 404 returned error can't find the container with id 7106e35d16b979cda061e0a0f6f5b85ad97d15e06bb2b6c33b0f8f50cf5ccc16 Aug 04 10:37:12.917526 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:12.917476 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2" event={"ID":"54995a67-600f-4bda-bbd0-c86307e0df08","Type":"ContainerStarted","Data":"08f5d539e118a52984e05487780c3c74dfdd33bde2de0b753469e1920ecd1b8b"} Aug 04 10:37:12.917882 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:12.917545 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2" event={"ID":"54995a67-600f-4bda-bbd0-c86307e0df08","Type":"ContainerStarted","Data":"7106e35d16b979cda061e0a0f6f5b85ad97d15e06bb2b6c33b0f8f50cf5ccc16"} Aug 04 10:37:12.938272 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:12.937937 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2" podStartSLOduration=1.777963471 podStartE2EDuration="1.937918976s" totalImagesPullingTime="159.955505ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:37:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:37:12.937059115 +0000 UTC m=+570.911141642" watchObservedRunningTime="2026-08-04 10:37:12.937918976 +0000 UTC m=+570.912001504" Aug 04 10:37:22.147178 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:22.147146 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2" Aug 04 10:37:22.147559 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:22.147204 2535 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2" Aug 04 10:37:22.159718 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:22.159694 2535 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2" Aug 04 10:37:22.974014 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:22.973988 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2" Aug 04 10:37:40.658635 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:40.658594 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:37:47.770697 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:37:47.770662 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:38:02.603564 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:38:02.603528 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:38:05.804219 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:38:05.804137 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:38:30.691903 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:38:30.691862 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:38:30.760141 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:38:30.760106 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:38:50.506272 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:38:50.506237 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:38:56.779115 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:38:56.779083 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:38:56.873806 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:38:56.873774 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:38:57.931227 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:38:57.931194 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:38:58.035505 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:38:58.035452 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:39:37.915448 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:39:37.915365 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:39:37.981033 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:39:37.981003 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:39:44.830150 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:39:44.830112 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67c8859cd8-8nx9w"] Aug 04 10:39:44.830619 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:39:44.830354 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-67c8859cd8-8nx9w" podUID="dda5dfce-9fdb-47c6-8d4b-faf00ec66182" containerName="manager" containerID="cri-o://1361f6d7a0e2e977e8d0cc9846a6b88571ee778d03d30ea999187d363498bdd4" gracePeriod=30 Aug 04 10:39:45.087586 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:39:45.087525 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67c8859cd8-8nx9w" Aug 04 10:39:45.246206 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:39:45.246169 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpgjb\" (UniqueName: \"kubernetes.io/projected/dda5dfce-9fdb-47c6-8d4b-faf00ec66182-kube-api-access-cpgjb\") pod \"dda5dfce-9fdb-47c6-8d4b-faf00ec66182\" (UID: \"dda5dfce-9fdb-47c6-8d4b-faf00ec66182\") " Aug 04 10:39:45.246363 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:39:45.246282 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dda5dfce-9fdb-47c6-8d4b-faf00ec66182-webhook-cert\") pod \"dda5dfce-9fdb-47c6-8d4b-faf00ec66182\" (UID: \"dda5dfce-9fdb-47c6-8d4b-faf00ec66182\") " Aug 04 10:39:45.248104 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:39:45.248071 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dda5dfce-9fdb-47c6-8d4b-faf00ec66182-kube-api-access-cpgjb" (OuterVolumeSpecName: "kube-api-access-cpgjb") pod "dda5dfce-9fdb-47c6-8d4b-faf00ec66182" (UID: "dda5dfce-9fdb-47c6-8d4b-faf00ec66182"). InnerVolumeSpecName "kube-api-access-cpgjb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:39:45.248223 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:39:45.248168 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dda5dfce-9fdb-47c6-8d4b-faf00ec66182-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "dda5dfce-9fdb-47c6-8d4b-faf00ec66182" (UID: "dda5dfce-9fdb-47c6-8d4b-faf00ec66182"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 10:39:45.347654 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:39:45.347586 2535 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dda5dfce-9fdb-47c6-8d4b-faf00ec66182-webhook-cert\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:39:45.347654 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:39:45.347609 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpgjb\" (UniqueName: \"kubernetes.io/projected/dda5dfce-9fdb-47c6-8d4b-faf00ec66182-kube-api-access-cpgjb\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:39:45.590381 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:39:45.590344 2535 generic.go:356] "Generic (PLEG): container finished" podID="dda5dfce-9fdb-47c6-8d4b-faf00ec66182" containerID="1361f6d7a0e2e977e8d0cc9846a6b88571ee778d03d30ea999187d363498bdd4" exitCode=0 Aug 04 10:39:45.590592 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:39:45.590390 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67c8859cd8-8nx9w" Aug 04 10:39:45.590592 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:39:45.590403 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67c8859cd8-8nx9w" event={"ID":"dda5dfce-9fdb-47c6-8d4b-faf00ec66182","Type":"ContainerDied","Data":"1361f6d7a0e2e977e8d0cc9846a6b88571ee778d03d30ea999187d363498bdd4"} Aug 04 10:39:45.590592 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:39:45.590450 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67c8859cd8-8nx9w" event={"ID":"dda5dfce-9fdb-47c6-8d4b-faf00ec66182","Type":"ContainerDied","Data":"62b112d381bfd3ef9e6c30f80c97d4d67feac916fe4667db0f1312ac774da6a7"} Aug 04 10:39:45.590592 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:39:45.590472 2535 scope.go:117] "RemoveContainer" containerID="1361f6d7a0e2e977e8d0cc9846a6b88571ee778d03d30ea999187d363498bdd4" Aug 04 10:39:45.599324 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:39:45.599306 2535 scope.go:117] "RemoveContainer" containerID="1361f6d7a0e2e977e8d0cc9846a6b88571ee778d03d30ea999187d363498bdd4" Aug 04 10:39:45.599572 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:39:45.599554 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1361f6d7a0e2e977e8d0cc9846a6b88571ee778d03d30ea999187d363498bdd4\": container with ID starting with 1361f6d7a0e2e977e8d0cc9846a6b88571ee778d03d30ea999187d363498bdd4 not found: ID does not exist" containerID="1361f6d7a0e2e977e8d0cc9846a6b88571ee778d03d30ea999187d363498bdd4" Aug 04 10:39:45.599633 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:39:45.599578 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1361f6d7a0e2e977e8d0cc9846a6b88571ee778d03d30ea999187d363498bdd4"} err="failed to get container status \"1361f6d7a0e2e977e8d0cc9846a6b88571ee778d03d30ea999187d363498bdd4\": rpc error: code = NotFound desc = could not find container \"1361f6d7a0e2e977e8d0cc9846a6b88571ee778d03d30ea999187d363498bdd4\": container with ID starting with 1361f6d7a0e2e977e8d0cc9846a6b88571ee778d03d30ea999187d363498bdd4 not found: ID does not exist" Aug 04 10:39:45.612302 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:39:45.612264 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67c8859cd8-8nx9w"] Aug 04 10:39:45.614764 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:39:45.614741 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-67c8859cd8-8nx9w"] Aug 04 10:39:46.480765 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:39:46.480726 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dda5dfce-9fdb-47c6-8d4b-faf00ec66182" path="/var/lib/kubelet/pods/dda5dfce-9fdb-47c6-8d4b-faf00ec66182/volumes" Aug 04 10:40:46.453216 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:40:46.453177 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67c8859cd8-kfnbw"] Aug 04 10:40:46.453646 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:40:46.453555 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dda5dfce-9fdb-47c6-8d4b-faf00ec66182" containerName="manager" Aug 04 10:40:46.453646 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:40:46.453566 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="dda5dfce-9fdb-47c6-8d4b-faf00ec66182" containerName="manager" Aug 04 10:40:46.453646 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:40:46.453622 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="dda5dfce-9fdb-47c6-8d4b-faf00ec66182" containerName="manager" Aug 04 10:40:46.456654 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:40:46.456638 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67c8859cd8-kfnbw" Aug 04 10:40:46.459538 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:40:46.459512 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Aug 04 10:40:46.459538 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:40:46.459530 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7tzq4\"" Aug 04 10:40:46.459738 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:40:46.459553 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Aug 04 10:40:46.470922 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:40:46.470899 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67c8859cd8-kfnbw"] Aug 04 10:40:46.478161 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:40:46.478141 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7dedadb0-d8a5-4f3a-ba75-3a3771c1816c-webhook-cert\") pod \"maas-controller-67c8859cd8-kfnbw\" (UID: \"7dedadb0-d8a5-4f3a-ba75-3a3771c1816c\") " pod="opendatahub/maas-controller-67c8859cd8-kfnbw" Aug 04 10:40:46.478362 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:40:46.478323 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fp2t\" (UniqueName: \"kubernetes.io/projected/7dedadb0-d8a5-4f3a-ba75-3a3771c1816c-kube-api-access-4fp2t\") pod \"maas-controller-67c8859cd8-kfnbw\" (UID: \"7dedadb0-d8a5-4f3a-ba75-3a3771c1816c\") " pod="opendatahub/maas-controller-67c8859cd8-kfnbw" Aug 04 10:40:46.578993 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:40:46.578963 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fp2t\" (UniqueName: \"kubernetes.io/projected/7dedadb0-d8a5-4f3a-ba75-3a3771c1816c-kube-api-access-4fp2t\") pod \"maas-controller-67c8859cd8-kfnbw\" (UID: \"7dedadb0-d8a5-4f3a-ba75-3a3771c1816c\") " pod="opendatahub/maas-controller-67c8859cd8-kfnbw" Aug 04 10:40:46.579159 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:40:46.579011 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7dedadb0-d8a5-4f3a-ba75-3a3771c1816c-webhook-cert\") pod \"maas-controller-67c8859cd8-kfnbw\" (UID: \"7dedadb0-d8a5-4f3a-ba75-3a3771c1816c\") " pod="opendatahub/maas-controller-67c8859cd8-kfnbw" Aug 04 10:40:46.581262 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:40:46.581233 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7dedadb0-d8a5-4f3a-ba75-3a3771c1816c-webhook-cert\") pod \"maas-controller-67c8859cd8-kfnbw\" (UID: \"7dedadb0-d8a5-4f3a-ba75-3a3771c1816c\") " pod="opendatahub/maas-controller-67c8859cd8-kfnbw" Aug 04 10:40:46.586465 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:40:46.586443 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fp2t\" (UniqueName: \"kubernetes.io/projected/7dedadb0-d8a5-4f3a-ba75-3a3771c1816c-kube-api-access-4fp2t\") pod \"maas-controller-67c8859cd8-kfnbw\" (UID: \"7dedadb0-d8a5-4f3a-ba75-3a3771c1816c\") " pod="opendatahub/maas-controller-67c8859cd8-kfnbw" Aug 04 10:40:46.766778 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:40:46.766704 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67c8859cd8-kfnbw" Aug 04 10:40:46.899821 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:40:46.899794 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67c8859cd8-kfnbw"] Aug 04 10:40:46.902530 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:40:46.902502 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dedadb0_d8a5_4f3a_ba75_3a3771c1816c.slice/crio-4c501ab9cceee085854f4b4a8fb4c6b6b3a171a6ab4fa40d777d4d908738c309 WatchSource:0}: Error finding container 4c501ab9cceee085854f4b4a8fb4c6b6b3a171a6ab4fa40d777d4d908738c309: Status 404 returned error can't find the container with id 4c501ab9cceee085854f4b4a8fb4c6b6b3a171a6ab4fa40d777d4d908738c309 Aug 04 10:40:46.903863 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:40:46.903841 2535 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 10:40:47.867005 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:40:47.866964 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67c8859cd8-kfnbw" event={"ID":"7dedadb0-d8a5-4f3a-ba75-3a3771c1816c","Type":"ContainerStarted","Data":"bef36d791a433040cfe9d292407aa32198027946216bcfcfe3d3a258a7b9b475"} Aug 04 10:40:47.867404 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:40:47.867125 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67c8859cd8-kfnbw" Aug 04 10:40:47.867404 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:40:47.867153 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67c8859cd8-kfnbw" event={"ID":"7dedadb0-d8a5-4f3a-ba75-3a3771c1816c","Type":"ContainerStarted","Data":"4c501ab9cceee085854f4b4a8fb4c6b6b3a171a6ab4fa40d777d4d908738c309"} Aug 04 10:40:47.887121 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:40:47.887072 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67c8859cd8-kfnbw" podStartSLOduration=1.421132808 podStartE2EDuration="1.887053905s" totalImagesPullingTime="465.921097ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:40:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:40:47.886174795 +0000 UTC m=+785.860257321" watchObservedRunningTime="2026-08-04 10:40:47.887053905 +0000 UTC m=+785.861136433" Aug 04 10:40:58.875055 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:40:58.875023 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67c8859cd8-kfnbw" Aug 04 10:41:03.438652 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:03.438617 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:41:03.544217 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:03.544182 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67c8859cd8-kfnbw"] Aug 04 10:41:03.544385 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:03.544366 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-67c8859cd8-kfnbw" podUID="7dedadb0-d8a5-4f3a-ba75-3a3771c1816c" containerName="manager" containerID="cri-o://bef36d791a433040cfe9d292407aa32198027946216bcfcfe3d3a258a7b9b475" gracePeriod=30 Aug 04 10:41:03.789044 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:03.789017 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67c8859cd8-kfnbw" Aug 04 10:41:03.916269 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:03.916237 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fp2t\" (UniqueName: \"kubernetes.io/projected/7dedadb0-d8a5-4f3a-ba75-3a3771c1816c-kube-api-access-4fp2t\") pod \"7dedadb0-d8a5-4f3a-ba75-3a3771c1816c\" (UID: \"7dedadb0-d8a5-4f3a-ba75-3a3771c1816c\") " Aug 04 10:41:03.916464 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:03.916280 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7dedadb0-d8a5-4f3a-ba75-3a3771c1816c-webhook-cert\") pod \"7dedadb0-d8a5-4f3a-ba75-3a3771c1816c\" (UID: \"7dedadb0-d8a5-4f3a-ba75-3a3771c1816c\") " Aug 04 10:41:03.918316 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:03.918274 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dedadb0-d8a5-4f3a-ba75-3a3771c1816c-kube-api-access-4fp2t" (OuterVolumeSpecName: "kube-api-access-4fp2t") pod "7dedadb0-d8a5-4f3a-ba75-3a3771c1816c" (UID: "7dedadb0-d8a5-4f3a-ba75-3a3771c1816c"). InnerVolumeSpecName "kube-api-access-4fp2t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:41:03.918421 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:03.918342 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7dedadb0-d8a5-4f3a-ba75-3a3771c1816c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "7dedadb0-d8a5-4f3a-ba75-3a3771c1816c" (UID: "7dedadb0-d8a5-4f3a-ba75-3a3771c1816c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 10:41:03.939323 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:03.939294 2535 generic.go:356] "Generic (PLEG): container finished" podID="7dedadb0-d8a5-4f3a-ba75-3a3771c1816c" containerID="bef36d791a433040cfe9d292407aa32198027946216bcfcfe3d3a258a7b9b475" exitCode=0 Aug 04 10:41:03.939461 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:03.939342 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67c8859cd8-kfnbw" Aug 04 10:41:03.939461 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:03.939347 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67c8859cd8-kfnbw" event={"ID":"7dedadb0-d8a5-4f3a-ba75-3a3771c1816c","Type":"ContainerDied","Data":"bef36d791a433040cfe9d292407aa32198027946216bcfcfe3d3a258a7b9b475"} Aug 04 10:41:03.939461 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:03.939421 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67c8859cd8-kfnbw" event={"ID":"7dedadb0-d8a5-4f3a-ba75-3a3771c1816c","Type":"ContainerDied","Data":"4c501ab9cceee085854f4b4a8fb4c6b6b3a171a6ab4fa40d777d4d908738c309"} Aug 04 10:41:03.939461 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:03.939451 2535 scope.go:117] "RemoveContainer" containerID="bef36d791a433040cfe9d292407aa32198027946216bcfcfe3d3a258a7b9b475" Aug 04 10:41:03.948280 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:03.948265 2535 scope.go:117] "RemoveContainer" containerID="bef36d791a433040cfe9d292407aa32198027946216bcfcfe3d3a258a7b9b475" Aug 04 10:41:03.948548 ip-10-0-130-250 kubenswrapper[2535]: E0804 10:41:03.948520 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bef36d791a433040cfe9d292407aa32198027946216bcfcfe3d3a258a7b9b475\": container with ID starting with bef36d791a433040cfe9d292407aa32198027946216bcfcfe3d3a258a7b9b475 not found: ID does not exist" containerID="bef36d791a433040cfe9d292407aa32198027946216bcfcfe3d3a258a7b9b475" Aug 04 10:41:03.948647 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:03.948555 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bef36d791a433040cfe9d292407aa32198027946216bcfcfe3d3a258a7b9b475"} err="failed to get container status \"bef36d791a433040cfe9d292407aa32198027946216bcfcfe3d3a258a7b9b475\": rpc error: code = NotFound desc = could not find container \"bef36d791a433040cfe9d292407aa32198027946216bcfcfe3d3a258a7b9b475\": container with ID starting with bef36d791a433040cfe9d292407aa32198027946216bcfcfe3d3a258a7b9b475 not found: ID does not exist" Aug 04 10:41:03.960588 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:03.960527 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67c8859cd8-kfnbw"] Aug 04 10:41:03.964178 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:03.964154 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-67c8859cd8-kfnbw"] Aug 04 10:41:04.017182 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:04.017152 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fp2t\" (UniqueName: \"kubernetes.io/projected/7dedadb0-d8a5-4f3a-ba75-3a3771c1816c-kube-api-access-4fp2t\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:41:04.017182 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:04.017178 2535 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7dedadb0-d8a5-4f3a-ba75-3a3771c1816c-webhook-cert\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:41:04.479789 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:04.479709 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7dedadb0-d8a5-4f3a-ba75-3a3771c1816c" path="/var/lib/kubelet/pods/7dedadb0-d8a5-4f3a-ba75-3a3771c1816c/volumes" Aug 04 10:41:04.662981 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:04.662948 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:41:04.781700 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:04.781671 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67c8859cd8-25fr9"] Aug 04 10:41:04.782036 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:04.782025 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dedadb0-d8a5-4f3a-ba75-3a3771c1816c" containerName="manager" Aug 04 10:41:04.782080 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:04.782038 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dedadb0-d8a5-4f3a-ba75-3a3771c1816c" containerName="manager" Aug 04 10:41:04.782119 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:04.782089 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="7dedadb0-d8a5-4f3a-ba75-3a3771c1816c" containerName="manager" Aug 04 10:41:04.786580 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:04.786561 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67c8859cd8-25fr9" Aug 04 10:41:04.789114 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:04.789097 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Aug 04 10:41:04.789224 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:04.789117 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7tzq4\"" Aug 04 10:41:04.789224 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:04.789129 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Aug 04 10:41:04.794747 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:04.794724 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67c8859cd8-25fr9"] Aug 04 10:41:04.825638 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:04.825601 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjx4x\" (UniqueName: \"kubernetes.io/projected/70003f72-bdcc-40b1-bf11-a7c546adc577-kube-api-access-pjx4x\") pod \"maas-controller-67c8859cd8-25fr9\" (UID: \"70003f72-bdcc-40b1-bf11-a7c546adc577\") " pod="opendatahub/maas-controller-67c8859cd8-25fr9" Aug 04 10:41:04.825638 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:04.825636 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/70003f72-bdcc-40b1-bf11-a7c546adc577-webhook-cert\") pod \"maas-controller-67c8859cd8-25fr9\" (UID: \"70003f72-bdcc-40b1-bf11-a7c546adc577\") " pod="opendatahub/maas-controller-67c8859cd8-25fr9" Aug 04 10:41:04.926279 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:04.926237 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjx4x\" (UniqueName: \"kubernetes.io/projected/70003f72-bdcc-40b1-bf11-a7c546adc577-kube-api-access-pjx4x\") pod \"maas-controller-67c8859cd8-25fr9\" (UID: \"70003f72-bdcc-40b1-bf11-a7c546adc577\") " pod="opendatahub/maas-controller-67c8859cd8-25fr9" Aug 04 10:41:04.926279 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:04.926285 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/70003f72-bdcc-40b1-bf11-a7c546adc577-webhook-cert\") pod \"maas-controller-67c8859cd8-25fr9\" (UID: \"70003f72-bdcc-40b1-bf11-a7c546adc577\") " pod="opendatahub/maas-controller-67c8859cd8-25fr9" Aug 04 10:41:04.928752 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:04.928722 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/70003f72-bdcc-40b1-bf11-a7c546adc577-webhook-cert\") pod \"maas-controller-67c8859cd8-25fr9\" (UID: \"70003f72-bdcc-40b1-bf11-a7c546adc577\") " pod="opendatahub/maas-controller-67c8859cd8-25fr9" Aug 04 10:41:04.936179 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:04.936158 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjx4x\" (UniqueName: \"kubernetes.io/projected/70003f72-bdcc-40b1-bf11-a7c546adc577-kube-api-access-pjx4x\") pod \"maas-controller-67c8859cd8-25fr9\" (UID: \"70003f72-bdcc-40b1-bf11-a7c546adc577\") " pod="opendatahub/maas-controller-67c8859cd8-25fr9" Aug 04 10:41:05.097839 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:05.097756 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67c8859cd8-25fr9" Aug 04 10:41:05.232904 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:05.232871 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67c8859cd8-25fr9"] Aug 04 10:41:05.235788 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:41:05.235755 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70003f72_bdcc_40b1_bf11_a7c546adc577.slice/crio-d30b23fe9dcb630bd8ce1990ab7501e674b9800e2006b18cad58e2a217cb6374 WatchSource:0}: Error finding container d30b23fe9dcb630bd8ce1990ab7501e674b9800e2006b18cad58e2a217cb6374: Status 404 returned error can't find the container with id d30b23fe9dcb630bd8ce1990ab7501e674b9800e2006b18cad58e2a217cb6374 Aug 04 10:41:05.950401 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:05.950375 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67c8859cd8-25fr9" event={"ID":"70003f72-bdcc-40b1-bf11-a7c546adc577","Type":"ContainerStarted","Data":"bc26663cfee892baa94e83de768eeca1890e18e43e9547770eafa66eee4fe607"} Aug 04 10:41:05.950722 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:05.950421 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67c8859cd8-25fr9" event={"ID":"70003f72-bdcc-40b1-bf11-a7c546adc577","Type":"ContainerStarted","Data":"d30b23fe9dcb630bd8ce1990ab7501e674b9800e2006b18cad58e2a217cb6374"} Aug 04 10:41:05.950722 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:05.950517 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67c8859cd8-25fr9" Aug 04 10:41:05.969150 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:05.969103 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67c8859cd8-25fr9" podStartSLOduration=1.313084377 podStartE2EDuration="1.969087213s" totalImagesPullingTime="656.002836ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:41:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:41:05.968350613 +0000 UTC m=+803.942433141" watchObservedRunningTime="2026-08-04 10:41:05.969087213 +0000 UTC m=+803.943169739" Aug 04 10:41:16.957850 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:16.957817 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67c8859cd8-25fr9" Aug 04 10:41:25.168368 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:25.168328 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:41:25.269534 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:41:25.269471 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:42:05.790686 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:42:05.790639 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:42:05.881877 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:42:05.881843 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:42:06.026880 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:42:06.026849 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:42:12.073699 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:42:12.073659 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:42:12.133255 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:42:12.133221 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:42:12.211163 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:42:12.211122 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:42:12.322788 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:42:12.322756 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:42:12.483914 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:42:12.483826 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:42:18.568007 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:42:18.567978 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:42:18.640083 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:42:18.640052 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:42:26.190882 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:42:26.190848 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:45:00.149916 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:45:00.149878 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764005-dpv9v"] Aug 04 10:45:00.153299 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:45:00.153282 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764005-dpv9v" Aug 04 10:45:00.155925 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:45:00.155905 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qptbt\"" Aug 04 10:45:00.162861 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:45:00.162843 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764005-dpv9v" Aug 04 10:45:00.168781 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:45:00.168751 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764005-dpv9v"] Aug 04 10:45:00.297529 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:45:00.297472 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764005-dpv9v"] Aug 04 10:45:00.300327 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:45:00.300300 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc4a02ed_caf3_4f3c_9235_b0fe6db253da.slice/crio-3adb0c9049303cc156c6f4e7314d848b67c9375f0b4f7b682d0143f217f522c1 WatchSource:0}: Error finding container 3adb0c9049303cc156c6f4e7314d848b67c9375f0b4f7b682d0143f217f522c1: Status 404 returned error can't find the container with id 3adb0c9049303cc156c6f4e7314d848b67c9375f0b4f7b682d0143f217f522c1 Aug 04 10:45:00.990559 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:45:00.990516 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764005-dpv9v" event={"ID":"fc4a02ed-caf3-4f3c-9235-b0fe6db253da","Type":"ContainerStarted","Data":"3adb0c9049303cc156c6f4e7314d848b67c9375f0b4f7b682d0143f217f522c1"} Aug 04 10:45:03.003598 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:45:03.003507 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764005-dpv9v" event={"ID":"fc4a02ed-caf3-4f3c-9235-b0fe6db253da","Type":"ContainerStarted","Data":"3b68c410078e2e7da293b2052332cfae62acb3d16e8ecf99b7fa35492258d666"} Aug 04 10:45:03.022385 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:45:03.022332 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764005-dpv9v" podStartSLOduration=0.992675952 podStartE2EDuration="3.022316353s" totalImagesPullingTime="2.029640401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:45:03.021508881 +0000 UTC m=+1040.995591433" watchObservedRunningTime="2026-08-04 10:45:03.022316353 +0000 UTC m=+1040.996398879" Aug 04 10:45:04.009197 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:45:04.009158 2535 generic.go:356] "Generic (PLEG): container finished" podID="fc4a02ed-caf3-4f3c-9235-b0fe6db253da" containerID="3b68c410078e2e7da293b2052332cfae62acb3d16e8ecf99b7fa35492258d666" exitCode=0 Aug 04 10:45:04.009654 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:45:04.009227 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764005-dpv9v" event={"ID":"fc4a02ed-caf3-4f3c-9235-b0fe6db253da","Type":"ContainerDied","Data":"3b68c410078e2e7da293b2052332cfae62acb3d16e8ecf99b7fa35492258d666"} Aug 04 10:45:05.144220 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:45:05.144194 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764005-dpv9v" Aug 04 10:45:06.020733 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:45:06.020696 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764005-dpv9v" event={"ID":"fc4a02ed-caf3-4f3c-9235-b0fe6db253da","Type":"ContainerDied","Data":"3adb0c9049303cc156c6f4e7314d848b67c9375f0b4f7b682d0143f217f522c1"} Aug 04 10:45:06.020894 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:45:06.020744 2535 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3adb0c9049303cc156c6f4e7314d848b67c9375f0b4f7b682d0143f217f522c1" Aug 04 10:45:06.020894 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:45:06.020710 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764005-dpv9v" Aug 04 10:45:11.625263 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:45:11.625226 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:45:14.221931 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:45:14.221865 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:45:24.098260 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:45:24.098225 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:51:06.579523 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:51:06.579468 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:51:35.565009 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:51:35.564931 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:51:42.850907 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:51:42.850871 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:52:37.331341 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:52:37.331299 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:52:37.862153 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:52:37.862116 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:53:06.489861 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:53:06.489782 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:53:06.590021 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:53:06.589989 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:53:27.531071 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:53:27.531034 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:56:07.477037 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:07.477003 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:56:12.756227 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:12.756171 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:56:38.791356 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:38.791320 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:56:41.990686 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:41.990652 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:56:42.085599 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:42.085568 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:56:42.428019 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:42.427989 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:56:48.740402 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:48.740363 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:56:48.841321 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:48.841291 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:56:48.996351 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:48.996268 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:56:49.326587 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:49.326542 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:56:49.584407 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:49.584325 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:56:51.260907 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:51.260863 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:56:51.357535 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:51.357481 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:56:52.393955 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:52.393911 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:56:52.598383 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:52.598349 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:56:52.928963 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:52.928916 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:56:53.027810 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:53.027768 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:56:53.034661 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:53.034632 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5"] Aug 04 10:56:53.035055 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:53.035038 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc4a02ed-caf3-4f3c-9235-b0fe6db253da" containerName="cleanup" Aug 04 10:56:53.035136 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:53.035057 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc4a02ed-caf3-4f3c-9235-b0fe6db253da" containerName="cleanup" Aug 04 10:56:53.035192 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:53.035138 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc4a02ed-caf3-4f3c-9235-b0fe6db253da" containerName="cleanup" Aug 04 10:56:53.038454 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:53.038435 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5" Aug 04 10:56:53.041580 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:53.041559 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-5f7d6dc8-dockercfg-vqrms\"" Aug 04 10:56:53.041827 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:53.041800 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-5f7d6dc8\"" Aug 04 10:56:53.048710 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:53.048688 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5"] Aug 04 10:56:53.065847 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:53.065825 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2"] Aug 04 10:56:53.069357 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:53.069341 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2" Aug 04 10:56:53.078313 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:53.078292 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2"] Aug 04 10:56:53.144737 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:53.144704 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/676bb90c-d4c1-40b0-82dd-3d1f8f33945c-plugins-config-volume\") pod \"payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2\" (UID: \"676bb90c-d4c1-40b0-82dd-3d1f8f33945c\") " pod="openshift-ingress/payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2" Aug 04 10:56:53.145012 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:53.144968 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l4gx\" (UniqueName: \"kubernetes.io/projected/676bb90c-d4c1-40b0-82dd-3d1f8f33945c-kube-api-access-7l4gx\") pod \"payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2\" (UID: \"676bb90c-d4c1-40b0-82dd-3d1f8f33945c\") " pod="openshift-ingress/payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2" Aug 04 10:56:53.145144 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:53.145071 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27w8t\" (UniqueName: \"kubernetes.io/projected/89efde97-e5c0-47bd-926b-3371905a7ef3-kube-api-access-27w8t\") pod \"payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5\" (UID: \"89efde97-e5c0-47bd-926b-3371905a7ef3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5" Aug 04 10:56:53.145215 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:53.145163 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89efde97-e5c0-47bd-926b-3371905a7ef3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5\" (UID: \"89efde97-e5c0-47bd-926b-3371905a7ef3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5" Aug 04 10:56:53.246668 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:53.246578 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89efde97-e5c0-47bd-926b-3371905a7ef3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5\" (UID: \"89efde97-e5c0-47bd-926b-3371905a7ef3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5" Aug 04 10:56:53.246825 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:53.246704 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/676bb90c-d4c1-40b0-82dd-3d1f8f33945c-plugins-config-volume\") pod \"payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2\" (UID: \"676bb90c-d4c1-40b0-82dd-3d1f8f33945c\") " pod="openshift-ingress/payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2" Aug 04 10:56:53.246825 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:53.246729 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l4gx\" (UniqueName: \"kubernetes.io/projected/676bb90c-d4c1-40b0-82dd-3d1f8f33945c-kube-api-access-7l4gx\") pod \"payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2\" (UID: \"676bb90c-d4c1-40b0-82dd-3d1f8f33945c\") " pod="openshift-ingress/payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2" Aug 04 10:56:53.246825 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:53.246780 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27w8t\" (UniqueName: \"kubernetes.io/projected/89efde97-e5c0-47bd-926b-3371905a7ef3-kube-api-access-27w8t\") pod \"payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5\" (UID: \"89efde97-e5c0-47bd-926b-3371905a7ef3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5" Aug 04 10:56:53.247394 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:53.247374 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/676bb90c-d4c1-40b0-82dd-3d1f8f33945c-plugins-config-volume\") pod \"payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2\" (UID: \"676bb90c-d4c1-40b0-82dd-3d1f8f33945c\") " pod="openshift-ingress/payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2" Aug 04 10:56:53.247447 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:53.247416 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89efde97-e5c0-47bd-926b-3371905a7ef3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5\" (UID: \"89efde97-e5c0-47bd-926b-3371905a7ef3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5" Aug 04 10:56:53.254811 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:53.254789 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27w8t\" (UniqueName: \"kubernetes.io/projected/89efde97-e5c0-47bd-926b-3371905a7ef3-kube-api-access-27w8t\") pod \"payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5\" (UID: \"89efde97-e5c0-47bd-926b-3371905a7ef3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5" Aug 04 10:56:53.254958 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:53.254886 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l4gx\" (UniqueName: \"kubernetes.io/projected/676bb90c-d4c1-40b0-82dd-3d1f8f33945c-kube-api-access-7l4gx\") pod \"payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2\" (UID: \"676bb90c-d4c1-40b0-82dd-3d1f8f33945c\") " pod="openshift-ingress/payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2" Aug 04 10:56:53.350136 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:53.350099 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5" Aug 04 10:56:53.380956 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:53.380925 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2" Aug 04 10:56:53.503702 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:53.503664 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5"] Aug 04 10:56:53.505902 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:56:53.505867 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89efde97_e5c0_47bd_926b_3371905a7ef3.slice/crio-6101a62943652fb5fe96ca4c6e60a686fd18c379b888ec6e21957a5f6de0c3c3 WatchSource:0}: Error finding container 6101a62943652fb5fe96ca4c6e60a686fd18c379b888ec6e21957a5f6de0c3c3: Status 404 returned error can't find the container with id 6101a62943652fb5fe96ca4c6e60a686fd18c379b888ec6e21957a5f6de0c3c3 Aug 04 10:56:53.533242 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:53.533220 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2"] Aug 04 10:56:53.541789 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:56:53.541746 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod676bb90c_d4c1_40b0_82dd_3d1f8f33945c.slice/crio-e126ed1df158ba0e1097172c28fb7d696c45fda832061e177c90442b8dadaa8d WatchSource:0}: Error finding container e126ed1df158ba0e1097172c28fb7d696c45fda832061e177c90442b8dadaa8d: Status 404 returned error can't find the container with id e126ed1df158ba0e1097172c28fb7d696c45fda832061e177c90442b8dadaa8d Aug 04 10:56:54.143531 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:54.143481 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5" event={"ID":"89efde97-e5c0-47bd-926b-3371905a7ef3","Type":"ContainerStarted","Data":"f049282b031e66d296236b0bcb85c5a7897c3ce254d847cd404e15ef8fda7ab7"} Aug 04 10:56:54.143736 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:54.143556 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5" event={"ID":"89efde97-e5c0-47bd-926b-3371905a7ef3","Type":"ContainerStarted","Data":"6101a62943652fb5fe96ca4c6e60a686fd18c379b888ec6e21957a5f6de0c3c3"} Aug 04 10:56:54.143736 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:54.143578 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5" Aug 04 10:56:54.144997 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:54.144973 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2" event={"ID":"676bb90c-d4c1-40b0-82dd-3d1f8f33945c","Type":"ContainerStarted","Data":"00b5dde5fe0915ad96cb5dbdbe08a6caa88c021609c3893b8fcf1c10e8c26b39"} Aug 04 10:56:54.145111 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:54.145007 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2" event={"ID":"676bb90c-d4c1-40b0-82dd-3d1f8f33945c","Type":"ContainerStarted","Data":"e126ed1df158ba0e1097172c28fb7d696c45fda832061e177c90442b8dadaa8d"} Aug 04 10:56:54.145111 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:54.145028 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2" Aug 04 10:56:54.162598 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:54.162555 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5" podStartSLOduration=1.16253692 podStartE2EDuration="1.16253692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:56:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:56:54.160769006 +0000 UTC m=+1752.134851556" watchObservedRunningTime="2026-08-04 10:56:54.16253692 +0000 UTC m=+1752.136619452" Aug 04 10:56:54.177687 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:56:54.177633 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2" podStartSLOduration=1.177616394 podStartE2EDuration="1.177616394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:56:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:56:54.175324282 +0000 UTC m=+1752.149406832" watchObservedRunningTime="2026-08-04 10:56:54.177616394 +0000 UTC m=+1752.151698924" Aug 04 10:57:01.704058 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:01.704020 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:57:05.151693 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:05.151648 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5" Aug 04 10:57:05.152062 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:05.151731 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2" Aug 04 10:57:23.111256 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:23.111215 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:57:23.129505 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:23.129455 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2"] Aug 04 10:57:23.129753 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:23.129717 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2" podUID="676bb90c-d4c1-40b0-82dd-3d1f8f33945c" containerName="payload-processing" containerID="cri-o://00b5dde5fe0915ad96cb5dbdbe08a6caa88c021609c3893b8fcf1c10e8c26b39" gracePeriod=30 Aug 04 10:57:23.136897 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:23.136873 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5"] Aug 04 10:57:23.137090 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:23.137065 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5" podUID="89efde97-e5c0-47bd-926b-3371905a7ef3" containerName="payload-pre-processing" containerID="cri-o://f049282b031e66d296236b0bcb85c5a7897c3ce254d847cd404e15ef8fda7ab7" gracePeriod=30 Aug 04 10:57:23.215535 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:23.215504 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:57:23.292731 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:23.292566 2535 generic.go:356] "Generic (PLEG): container finished" podID="89efde97-e5c0-47bd-926b-3371905a7ef3" containerID="f049282b031e66d296236b0bcb85c5a7897c3ce254d847cd404e15ef8fda7ab7" exitCode=0 Aug 04 10:57:23.292731 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:23.292665 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5" event={"ID":"89efde97-e5c0-47bd-926b-3371905a7ef3","Type":"ContainerDied","Data":"f049282b031e66d296236b0bcb85c5a7897c3ce254d847cd404e15ef8fda7ab7"} Aug 04 10:57:23.294418 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:23.294368 2535 generic.go:356] "Generic (PLEG): container finished" podID="676bb90c-d4c1-40b0-82dd-3d1f8f33945c" containerID="00b5dde5fe0915ad96cb5dbdbe08a6caa88c021609c3893b8fcf1c10e8c26b39" exitCode=0 Aug 04 10:57:23.294570 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:23.294481 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2" event={"ID":"676bb90c-d4c1-40b0-82dd-3d1f8f33945c","Type":"ContainerDied","Data":"00b5dde5fe0915ad96cb5dbdbe08a6caa88c021609c3893b8fcf1c10e8c26b39"} Aug 04 10:57:23.316877 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:23.316807 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:57:23.413846 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:23.413824 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2" Aug 04 10:57:23.419293 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:23.419269 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5" Aug 04 10:57:23.517235 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:23.517206 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:57:23.520261 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:23.520239 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89efde97-e5c0-47bd-926b-3371905a7ef3-plugins-config-volume\") pod \"89efde97-e5c0-47bd-926b-3371905a7ef3\" (UID: \"89efde97-e5c0-47bd-926b-3371905a7ef3\") " Aug 04 10:57:23.520388 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:23.520273 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/676bb90c-d4c1-40b0-82dd-3d1f8f33945c-plugins-config-volume\") pod \"676bb90c-d4c1-40b0-82dd-3d1f8f33945c\" (UID: \"676bb90c-d4c1-40b0-82dd-3d1f8f33945c\") " Aug 04 10:57:23.520388 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:23.520314 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7l4gx\" (UniqueName: \"kubernetes.io/projected/676bb90c-d4c1-40b0-82dd-3d1f8f33945c-kube-api-access-7l4gx\") pod \"676bb90c-d4c1-40b0-82dd-3d1f8f33945c\" (UID: \"676bb90c-d4c1-40b0-82dd-3d1f8f33945c\") " Aug 04 10:57:23.520388 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:23.520361 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27w8t\" (UniqueName: \"kubernetes.io/projected/89efde97-e5c0-47bd-926b-3371905a7ef3-kube-api-access-27w8t\") pod \"89efde97-e5c0-47bd-926b-3371905a7ef3\" (UID: \"89efde97-e5c0-47bd-926b-3371905a7ef3\") " Aug 04 10:57:23.520692 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:23.520664 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89efde97-e5c0-47bd-926b-3371905a7ef3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "89efde97-e5c0-47bd-926b-3371905a7ef3" (UID: "89efde97-e5c0-47bd-926b-3371905a7ef3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:57:23.520813 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:23.520742 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/676bb90c-d4c1-40b0-82dd-3d1f8f33945c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "676bb90c-d4c1-40b0-82dd-3d1f8f33945c" (UID: "676bb90c-d4c1-40b0-82dd-3d1f8f33945c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:57:23.522342 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:23.522321 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/676bb90c-d4c1-40b0-82dd-3d1f8f33945c-kube-api-access-7l4gx" (OuterVolumeSpecName: "kube-api-access-7l4gx") pod "676bb90c-d4c1-40b0-82dd-3d1f8f33945c" (UID: "676bb90c-d4c1-40b0-82dd-3d1f8f33945c"). InnerVolumeSpecName "kube-api-access-7l4gx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:57:23.522401 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:23.522372 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89efde97-e5c0-47bd-926b-3371905a7ef3-kube-api-access-27w8t" (OuterVolumeSpecName: "kube-api-access-27w8t") pod "89efde97-e5c0-47bd-926b-3371905a7ef3" (UID: "89efde97-e5c0-47bd-926b-3371905a7ef3"). InnerVolumeSpecName "kube-api-access-27w8t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:57:23.621045 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:23.621016 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89efde97-e5c0-47bd-926b-3371905a7ef3-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:57:23.621045 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:23.621041 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/676bb90c-d4c1-40b0-82dd-3d1f8f33945c-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:57:23.621045 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:23.621051 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7l4gx\" (UniqueName: \"kubernetes.io/projected/676bb90c-d4c1-40b0-82dd-3d1f8f33945c-kube-api-access-7l4gx\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:57:23.621259 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:23.621061 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27w8t\" (UniqueName: \"kubernetes.io/projected/89efde97-e5c0-47bd-926b-3371905a7ef3-kube-api-access-27w8t\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:57:23.741590 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:23.741507 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:57:23.959980 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:23.959945 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:57:24.060299 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:24.060271 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:57:24.200649 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:24.200620 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:57:24.300904 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:24.300865 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5" Aug 04 10:57:24.301084 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:24.300863 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5" event={"ID":"89efde97-e5c0-47bd-926b-3371905a7ef3","Type":"ContainerDied","Data":"6101a62943652fb5fe96ca4c6e60a686fd18c379b888ec6e21957a5f6de0c3c3"} Aug 04 10:57:24.301084 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:24.301051 2535 scope.go:117] "RemoveContainer" containerID="f049282b031e66d296236b0bcb85c5a7897c3ce254d847cd404e15ef8fda7ab7" Aug 04 10:57:24.302543 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:24.302325 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2" event={"ID":"676bb90c-d4c1-40b0-82dd-3d1f8f33945c","Type":"ContainerDied","Data":"e126ed1df158ba0e1097172c28fb7d696c45fda832061e177c90442b8dadaa8d"} Aug 04 10:57:24.302543 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:24.302350 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2" Aug 04 10:57:24.309849 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:24.309804 2535 scope.go:117] "RemoveContainer" containerID="00b5dde5fe0915ad96cb5dbdbe08a6caa88c021609c3893b8fcf1c10e8c26b39" Aug 04 10:57:24.325779 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:24.325757 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5"] Aug 04 10:57:24.332804 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:24.332786 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5f7d6dc8-866948599b-gt6q5"] Aug 04 10:57:24.343743 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:24.343720 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2"] Aug 04 10:57:24.346652 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:24.346625 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5f7d6dc8-77ccc58ff4-5bcb2"] Aug 04 10:57:24.488474 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:24.488404 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="676bb90c-d4c1-40b0-82dd-3d1f8f33945c" path="/var/lib/kubelet/pods/676bb90c-d4c1-40b0-82dd-3d1f8f33945c/volumes" Aug 04 10:57:24.489014 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:24.488979 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89efde97-e5c0-47bd-926b-3371905a7ef3" path="/var/lib/kubelet/pods/89efde97-e5c0-47bd-926b-3371905a7ef3/volumes" Aug 04 10:57:24.547377 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:24.547338 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:57:24.776220 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:24.776188 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:57:26.569586 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:26.569548 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:57:26.713366 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:26.713330 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:57:26.855344 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:26.855265 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:57:27.292229 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.292198 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:57:27.384861 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.384826 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:57:27.455824 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.455791 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd"] Aug 04 10:57:27.456160 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.456147 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89efde97-e5c0-47bd-926b-3371905a7ef3" containerName="payload-pre-processing" Aug 04 10:57:27.456209 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.456163 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="89efde97-e5c0-47bd-926b-3371905a7ef3" containerName="payload-pre-processing" Aug 04 10:57:27.456209 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.456182 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="676bb90c-d4c1-40b0-82dd-3d1f8f33945c" containerName="payload-processing" Aug 04 10:57:27.456209 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.456188 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="676bb90c-d4c1-40b0-82dd-3d1f8f33945c" containerName="payload-processing" Aug 04 10:57:27.456302 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.456246 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="89efde97-e5c0-47bd-926b-3371905a7ef3" containerName="payload-pre-processing" Aug 04 10:57:27.456302 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.456255 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="676bb90c-d4c1-40b0-82dd-3d1f8f33945c" containerName="payload-processing" Aug 04 10:57:27.460911 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.460896 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd" Aug 04 10:57:27.463417 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.463392 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-03e71fbe\"" Aug 04 10:57:27.463561 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.463465 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-03e71fbe-dockercfg-zrbmh\"" Aug 04 10:57:27.468581 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.468478 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd"] Aug 04 10:57:27.556572 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.556467 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg26r\" (UniqueName: \"kubernetes.io/projected/7666f21e-ed02-4c02-acd1-a50207f4ae45-kube-api-access-cg26r\") pod \"payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd\" (UID: \"7666f21e-ed02-4c02-acd1-a50207f4ae45\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd" Aug 04 10:57:27.556751 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.556574 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7666f21e-ed02-4c02-acd1-a50207f4ae45-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd\" (UID: \"7666f21e-ed02-4c02-acd1-a50207f4ae45\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd" Aug 04 10:57:27.558467 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.558441 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4"] Aug 04 10:57:27.561808 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.561790 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4" Aug 04 10:57:27.573555 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.573530 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4"] Aug 04 10:57:27.657399 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.657372 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvqqr\" (UniqueName: \"kubernetes.io/projected/4f205b57-2681-44c5-8d6a-a6300dfef98f-kube-api-access-hvqqr\") pod \"payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4\" (UID: \"4f205b57-2681-44c5-8d6a-a6300dfef98f\") " pod="openshift-ingress/payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4" Aug 04 10:57:27.657559 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.657449 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg26r\" (UniqueName: \"kubernetes.io/projected/7666f21e-ed02-4c02-acd1-a50207f4ae45-kube-api-access-cg26r\") pod \"payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd\" (UID: \"7666f21e-ed02-4c02-acd1-a50207f4ae45\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd" Aug 04 10:57:27.657559 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.657482 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f205b57-2681-44c5-8d6a-a6300dfef98f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4\" (UID: \"4f205b57-2681-44c5-8d6a-a6300dfef98f\") " pod="openshift-ingress/payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4" Aug 04 10:57:27.657559 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.657542 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7666f21e-ed02-4c02-acd1-a50207f4ae45-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd\" (UID: \"7666f21e-ed02-4c02-acd1-a50207f4ae45\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd" Aug 04 10:57:27.658155 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.658137 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7666f21e-ed02-4c02-acd1-a50207f4ae45-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd\" (UID: \"7666f21e-ed02-4c02-acd1-a50207f4ae45\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd" Aug 04 10:57:27.665234 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.665208 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg26r\" (UniqueName: \"kubernetes.io/projected/7666f21e-ed02-4c02-acd1-a50207f4ae45-kube-api-access-cg26r\") pod \"payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd\" (UID: \"7666f21e-ed02-4c02-acd1-a50207f4ae45\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd" Aug 04 10:57:27.758152 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.758123 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvqqr\" (UniqueName: \"kubernetes.io/projected/4f205b57-2681-44c5-8d6a-a6300dfef98f-kube-api-access-hvqqr\") pod \"payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4\" (UID: \"4f205b57-2681-44c5-8d6a-a6300dfef98f\") " pod="openshift-ingress/payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4" Aug 04 10:57:27.758313 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.758182 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f205b57-2681-44c5-8d6a-a6300dfef98f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4\" (UID: \"4f205b57-2681-44c5-8d6a-a6300dfef98f\") " pod="openshift-ingress/payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4" Aug 04 10:57:27.758865 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.758848 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f205b57-2681-44c5-8d6a-a6300dfef98f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4\" (UID: \"4f205b57-2681-44c5-8d6a-a6300dfef98f\") " pod="openshift-ingress/payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4" Aug 04 10:57:27.765863 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.765840 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvqqr\" (UniqueName: \"kubernetes.io/projected/4f205b57-2681-44c5-8d6a-a6300dfef98f-kube-api-access-hvqqr\") pod \"payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4\" (UID: \"4f205b57-2681-44c5-8d6a-a6300dfef98f\") " pod="openshift-ingress/payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4" Aug 04 10:57:27.773658 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.773625 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd" Aug 04 10:57:27.874820 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.874727 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4" Aug 04 10:57:27.914760 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:27.914735 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd"] Aug 04 10:57:27.917748 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:57:27.917719 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7666f21e_ed02_4c02_acd1_a50207f4ae45.slice/crio-c38a78d1d5201bff2b1489270bac6cf2302defd81266b0ccb7a39ccfda5c2c79 WatchSource:0}: Error finding container c38a78d1d5201bff2b1489270bac6cf2302defd81266b0ccb7a39ccfda5c2c79: Status 404 returned error can't find the container with id c38a78d1d5201bff2b1489270bac6cf2302defd81266b0ccb7a39ccfda5c2c79 Aug 04 10:57:28.027750 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:28.027727 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4"] Aug 04 10:57:28.029406 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:57:28.029379 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f205b57_2681_44c5_8d6a_a6300dfef98f.slice/crio-c6304170a150a80f543ad716d92e2ae83bad9b1de063ec736b06e8addcb7018f WatchSource:0}: Error finding container c6304170a150a80f543ad716d92e2ae83bad9b1de063ec736b06e8addcb7018f: Status 404 returned error can't find the container with id c6304170a150a80f543ad716d92e2ae83bad9b1de063ec736b06e8addcb7018f Aug 04 10:57:28.325177 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:28.325141 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd" event={"ID":"7666f21e-ed02-4c02-acd1-a50207f4ae45","Type":"ContainerStarted","Data":"113a9667d5e702b51bb739677e779e46ab8164779f3e253866eb65f19bb4836b"} Aug 04 10:57:28.325378 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:28.325200 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd" event={"ID":"7666f21e-ed02-4c02-acd1-a50207f4ae45","Type":"ContainerStarted","Data":"c38a78d1d5201bff2b1489270bac6cf2302defd81266b0ccb7a39ccfda5c2c79"} Aug 04 10:57:28.325378 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:28.325225 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd" Aug 04 10:57:28.326627 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:28.326567 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4" event={"ID":"4f205b57-2681-44c5-8d6a-a6300dfef98f","Type":"ContainerStarted","Data":"6f4d5d461ad99d559444b9a2f9aaa447375fcce33e6ff5588e29dd60aae237e3"} Aug 04 10:57:28.326627 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:28.326607 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4" event={"ID":"4f205b57-2681-44c5-8d6a-a6300dfef98f","Type":"ContainerStarted","Data":"c6304170a150a80f543ad716d92e2ae83bad9b1de063ec736b06e8addcb7018f"} Aug 04 10:57:28.326762 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:28.326628 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4" Aug 04 10:57:28.342249 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:28.342197 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd" podStartSLOduration=1.342178138 podStartE2EDuration="1.342178138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:57:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:57:28.341149224 +0000 UTC m=+1786.315231750" watchObservedRunningTime="2026-08-04 10:57:28.342178138 +0000 UTC m=+1786.316260733" Aug 04 10:57:28.359247 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:28.359204 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4" podStartSLOduration=1.359190135 podStartE2EDuration="1.359190135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:57:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:57:28.356726076 +0000 UTC m=+1786.330808603" watchObservedRunningTime="2026-08-04 10:57:28.359190135 +0000 UTC m=+1786.333272661" Aug 04 10:57:39.333568 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:39.333480 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd" Aug 04 10:57:39.334003 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:39.333568 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4" Aug 04 10:57:58.257353 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:58.257313 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:57:58.282258 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:58.282222 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4"] Aug 04 10:57:58.282536 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:58.282509 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4" podUID="4f205b57-2681-44c5-8d6a-a6300dfef98f" containerName="payload-processing" containerID="cri-o://6f4d5d461ad99d559444b9a2f9aaa447375fcce33e6ff5588e29dd60aae237e3" gracePeriod=30 Aug 04 10:57:58.290168 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:58.290140 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd"] Aug 04 10:57:58.290983 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:58.290579 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd" podUID="7666f21e-ed02-4c02-acd1-a50207f4ae45" containerName="payload-pre-processing" containerID="cri-o://113a9667d5e702b51bb739677e779e46ab8164779f3e253866eb65f19bb4836b" gracePeriod=30 Aug 04 10:57:58.324504 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:58.324456 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:57:58.391107 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:58.391072 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:57:58.452168 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:58.451954 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:57:58.468264 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:58.468225 2535 generic.go:356] "Generic (PLEG): container finished" podID="4f205b57-2681-44c5-8d6a-a6300dfef98f" containerID="6f4d5d461ad99d559444b9a2f9aaa447375fcce33e6ff5588e29dd60aae237e3" exitCode=0 Aug 04 10:57:58.468406 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:58.468293 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4" event={"ID":"4f205b57-2681-44c5-8d6a-a6300dfef98f","Type":"ContainerDied","Data":"6f4d5d461ad99d559444b9a2f9aaa447375fcce33e6ff5588e29dd60aae237e3"} Aug 04 10:57:58.470370 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:58.470344 2535 generic.go:356] "Generic (PLEG): container finished" podID="7666f21e-ed02-4c02-acd1-a50207f4ae45" containerID="113a9667d5e702b51bb739677e779e46ab8164779f3e253866eb65f19bb4836b" exitCode=0 Aug 04 10:57:58.470467 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:58.470365 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd" event={"ID":"7666f21e-ed02-4c02-acd1-a50207f4ae45","Type":"ContainerDied","Data":"113a9667d5e702b51bb739677e779e46ab8164779f3e253866eb65f19bb4836b"} Aug 04 10:57:58.550574 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:58.549994 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4" Aug 04 10:57:58.574350 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:58.574327 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd" Aug 04 10:57:58.603718 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:58.603686 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:57:58.644876 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:58.644850 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7666f21e-ed02-4c02-acd1-a50207f4ae45-plugins-config-volume\") pod \"7666f21e-ed02-4c02-acd1-a50207f4ae45\" (UID: \"7666f21e-ed02-4c02-acd1-a50207f4ae45\") " Aug 04 10:57:58.645040 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:58.644910 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f205b57-2681-44c5-8d6a-a6300dfef98f-plugins-config-volume\") pod \"4f205b57-2681-44c5-8d6a-a6300dfef98f\" (UID: \"4f205b57-2681-44c5-8d6a-a6300dfef98f\") " Aug 04 10:57:58.645040 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:58.644943 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cg26r\" (UniqueName: \"kubernetes.io/projected/7666f21e-ed02-4c02-acd1-a50207f4ae45-kube-api-access-cg26r\") pod \"7666f21e-ed02-4c02-acd1-a50207f4ae45\" (UID: \"7666f21e-ed02-4c02-acd1-a50207f4ae45\") " Aug 04 10:57:58.645040 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:58.644959 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvqqr\" (UniqueName: \"kubernetes.io/projected/4f205b57-2681-44c5-8d6a-a6300dfef98f-kube-api-access-hvqqr\") pod \"4f205b57-2681-44c5-8d6a-a6300dfef98f\" (UID: \"4f205b57-2681-44c5-8d6a-a6300dfef98f\") " Aug 04 10:57:58.645251 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:58.645228 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7666f21e-ed02-4c02-acd1-a50207f4ae45-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7666f21e-ed02-4c02-acd1-a50207f4ae45" (UID: "7666f21e-ed02-4c02-acd1-a50207f4ae45"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:57:58.645359 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:58.645335 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f205b57-2681-44c5-8d6a-a6300dfef98f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4f205b57-2681-44c5-8d6a-a6300dfef98f" (UID: "4f205b57-2681-44c5-8d6a-a6300dfef98f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:57:58.647011 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:58.646988 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f205b57-2681-44c5-8d6a-a6300dfef98f-kube-api-access-hvqqr" (OuterVolumeSpecName: "kube-api-access-hvqqr") pod "4f205b57-2681-44c5-8d6a-a6300dfef98f" (UID: "4f205b57-2681-44c5-8d6a-a6300dfef98f"). InnerVolumeSpecName "kube-api-access-hvqqr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:57:58.647011 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:58.647000 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7666f21e-ed02-4c02-acd1-a50207f4ae45-kube-api-access-cg26r" (OuterVolumeSpecName: "kube-api-access-cg26r") pod "7666f21e-ed02-4c02-acd1-a50207f4ae45" (UID: "7666f21e-ed02-4c02-acd1-a50207f4ae45"). InnerVolumeSpecName "kube-api-access-cg26r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:57:58.746396 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:58.746366 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7666f21e-ed02-4c02-acd1-a50207f4ae45-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:57:58.746396 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:58.746392 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f205b57-2681-44c5-8d6a-a6300dfef98f-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:57:58.746396 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:58.746403 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cg26r\" (UniqueName: \"kubernetes.io/projected/7666f21e-ed02-4c02-acd1-a50207f4ae45-kube-api-access-cg26r\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:57:58.746632 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:58.746412 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvqqr\" (UniqueName: \"kubernetes.io/projected/4f205b57-2681-44c5-8d6a-a6300dfef98f-kube-api-access-hvqqr\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:57:58.891296 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:58.891211 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:57:59.144296 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:59.144207 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:57:59.478445 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:59.478366 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd" Aug 04 10:57:59.478903 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:59.478564 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd" event={"ID":"7666f21e-ed02-4c02-acd1-a50207f4ae45","Type":"ContainerDied","Data":"c38a78d1d5201bff2b1489270bac6cf2302defd81266b0ccb7a39ccfda5c2c79"} Aug 04 10:57:59.478903 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:59.478632 2535 scope.go:117] "RemoveContainer" containerID="113a9667d5e702b51bb739677e779e46ab8164779f3e253866eb65f19bb4836b" Aug 04 10:57:59.480568 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:59.480405 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4" event={"ID":"4f205b57-2681-44c5-8d6a-a6300dfef98f","Type":"ContainerDied","Data":"c6304170a150a80f543ad716d92e2ae83bad9b1de063ec736b06e8addcb7018f"} Aug 04 10:57:59.480568 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:59.480520 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4" Aug 04 10:57:59.493594 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:59.493246 2535 scope.go:117] "RemoveContainer" containerID="6f4d5d461ad99d559444b9a2f9aaa447375fcce33e6ff5588e29dd60aae237e3" Aug 04 10:57:59.527429 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:59.527402 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4"] Aug 04 10:57:59.547406 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:59.547385 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:57:59.551832 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:59.551811 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-03e71fbe-9f6b4cbf7-mmrj4"] Aug 04 10:57:59.566629 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:59.566610 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd"] Aug 04 10:57:59.578852 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:59.578830 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-03e71fbe-8456cc9b44-27xvd"] Aug 04 10:57:59.718425 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:57:59.718365 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:58:00.479811 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:00.479779 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f205b57-2681-44c5-8d6a-a6300dfef98f" path="/var/lib/kubelet/pods/4f205b57-2681-44c5-8d6a-a6300dfef98f/volumes" Aug 04 10:58:00.480159 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:00.480142 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7666f21e-ed02-4c02-acd1-a50207f4ae45" path="/var/lib/kubelet/pods/7666f21e-ed02-4c02-acd1-a50207f4ae45/volumes" Aug 04 10:58:01.637673 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:01.637639 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:58:01.732651 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:01.732590 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:58:02.768271 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:02.768239 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:58:02.861067 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:02.861037 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:58:03.377305 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.377275 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:58:03.478871 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.478816 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:58:03.484458 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.484429 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b"] Aug 04 10:58:03.484794 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.484779 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f205b57-2681-44c5-8d6a-a6300dfef98f" containerName="payload-processing" Aug 04 10:58:03.484794 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.484793 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f205b57-2681-44c5-8d6a-a6300dfef98f" containerName="payload-processing" Aug 04 10:58:03.484920 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.484802 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7666f21e-ed02-4c02-acd1-a50207f4ae45" containerName="payload-pre-processing" Aug 04 10:58:03.484920 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.484808 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="7666f21e-ed02-4c02-acd1-a50207f4ae45" containerName="payload-pre-processing" Aug 04 10:58:03.484920 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.484861 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f205b57-2681-44c5-8d6a-a6300dfef98f" containerName="payload-processing" Aug 04 10:58:03.484920 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.484870 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="7666f21e-ed02-4c02-acd1-a50207f4ae45" containerName="payload-pre-processing" Aug 04 10:58:03.489098 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.489079 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b" Aug 04 10:58:03.491536 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.491519 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-fb58dd84\"" Aug 04 10:58:03.499875 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.499855 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b"] Aug 04 10:58:03.515155 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.515132 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg"] Aug 04 10:58:03.518379 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.518359 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg" Aug 04 10:58:03.520421 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.520404 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-fb58dd84-dockercfg-h872b\"" Aug 04 10:58:03.526987 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.526967 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg"] Aug 04 10:58:03.588406 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.588371 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79188947-e71a-4461-aa66-0b3d1997ae13-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b\" (UID: \"79188947-e71a-4461-aa66-0b3d1997ae13\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b" Aug 04 10:58:03.588606 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.588439 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lkkm\" (UniqueName: \"kubernetes.io/projected/79188947-e71a-4461-aa66-0b3d1997ae13-kube-api-access-7lkkm\") pod \"payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b\" (UID: \"79188947-e71a-4461-aa66-0b3d1997ae13\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b" Aug 04 10:58:03.588606 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.588471 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2e41ff2-e104-4255-88a0-87c86fe2c501-plugins-config-volume\") pod \"payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg\" (UID: \"f2e41ff2-e104-4255-88a0-87c86fe2c501\") " pod="openshift-ingress/payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg" Aug 04 10:58:03.588606 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.588562 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7mz7\" (UniqueName: \"kubernetes.io/projected/f2e41ff2-e104-4255-88a0-87c86fe2c501-kube-api-access-r7mz7\") pod \"payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg\" (UID: \"f2e41ff2-e104-4255-88a0-87c86fe2c501\") " pod="openshift-ingress/payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg" Aug 04 10:58:03.689970 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.689880 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79188947-e71a-4461-aa66-0b3d1997ae13-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b\" (UID: \"79188947-e71a-4461-aa66-0b3d1997ae13\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b" Aug 04 10:58:03.689970 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.689931 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lkkm\" (UniqueName: \"kubernetes.io/projected/79188947-e71a-4461-aa66-0b3d1997ae13-kube-api-access-7lkkm\") pod \"payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b\" (UID: \"79188947-e71a-4461-aa66-0b3d1997ae13\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b" Aug 04 10:58:03.689970 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.689953 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2e41ff2-e104-4255-88a0-87c86fe2c501-plugins-config-volume\") pod \"payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg\" (UID: \"f2e41ff2-e104-4255-88a0-87c86fe2c501\") " pod="openshift-ingress/payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg" Aug 04 10:58:03.689970 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.689970 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7mz7\" (UniqueName: \"kubernetes.io/projected/f2e41ff2-e104-4255-88a0-87c86fe2c501-kube-api-access-r7mz7\") pod \"payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg\" (UID: \"f2e41ff2-e104-4255-88a0-87c86fe2c501\") " pod="openshift-ingress/payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg" Aug 04 10:58:03.690639 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.690612 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2e41ff2-e104-4255-88a0-87c86fe2c501-plugins-config-volume\") pod \"payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg\" (UID: \"f2e41ff2-e104-4255-88a0-87c86fe2c501\") " pod="openshift-ingress/payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg" Aug 04 10:58:03.690811 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.690790 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79188947-e71a-4461-aa66-0b3d1997ae13-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b\" (UID: \"79188947-e71a-4461-aa66-0b3d1997ae13\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b" Aug 04 10:58:03.697958 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.697930 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7mz7\" (UniqueName: \"kubernetes.io/projected/f2e41ff2-e104-4255-88a0-87c86fe2c501-kube-api-access-r7mz7\") pod \"payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg\" (UID: \"f2e41ff2-e104-4255-88a0-87c86fe2c501\") " pod="openshift-ingress/payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg" Aug 04 10:58:03.698052 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.697972 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lkkm\" (UniqueName: \"kubernetes.io/projected/79188947-e71a-4461-aa66-0b3d1997ae13-kube-api-access-7lkkm\") pod \"payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b\" (UID: \"79188947-e71a-4461-aa66-0b3d1997ae13\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b" Aug 04 10:58:03.799988 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.799946 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b" Aug 04 10:58:03.829588 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.829563 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg" Aug 04 10:58:03.965062 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.965031 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b"] Aug 04 10:58:03.969814 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:58:03.969758 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79188947_e71a_4461_aa66_0b3d1997ae13.slice/crio-167b4698cc0518b841e1d0aea4b2c51b887a8e0f3003a7541d444f5ebc3704e2 WatchSource:0}: Error finding container 167b4698cc0518b841e1d0aea4b2c51b887a8e0f3003a7541d444f5ebc3704e2: Status 404 returned error can't find the container with id 167b4698cc0518b841e1d0aea4b2c51b887a8e0f3003a7541d444f5ebc3704e2 Aug 04 10:58:03.999528 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:03.999472 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg"] Aug 04 10:58:04.001834 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:58:04.001808 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2e41ff2_e104_4255_88a0_87c86fe2c501.slice/crio-7d27c0f66d9dc724baf5165e2d4fbe56bc7ecfde905551203d98ea87869c018e WatchSource:0}: Error finding container 7d27c0f66d9dc724baf5165e2d4fbe56bc7ecfde905551203d98ea87869c018e: Status 404 returned error can't find the container with id 7d27c0f66d9dc724baf5165e2d4fbe56bc7ecfde905551203d98ea87869c018e Aug 04 10:58:04.506458 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:04.506425 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b" event={"ID":"79188947-e71a-4461-aa66-0b3d1997ae13","Type":"ContainerStarted","Data":"a6543c4adff69cc0dde7ae6a0dc7b7680348646a1498efaeff5d1cf721967d95"} Aug 04 10:58:04.506693 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:04.506481 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b" event={"ID":"79188947-e71a-4461-aa66-0b3d1997ae13","Type":"ContainerStarted","Data":"167b4698cc0518b841e1d0aea4b2c51b887a8e0f3003a7541d444f5ebc3704e2"} Aug 04 10:58:04.506693 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:04.506526 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b" Aug 04 10:58:04.508086 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:04.508060 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg" event={"ID":"f2e41ff2-e104-4255-88a0-87c86fe2c501","Type":"ContainerStarted","Data":"91c90c5c39c6ed76e357efa69150687d4a31276b0898b2d62283abbfa3651197"} Aug 04 10:58:04.508205 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:04.508106 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg" event={"ID":"f2e41ff2-e104-4255-88a0-87c86fe2c501","Type":"ContainerStarted","Data":"7d27c0f66d9dc724baf5165e2d4fbe56bc7ecfde905551203d98ea87869c018e"} Aug 04 10:58:04.508205 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:04.508136 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg" Aug 04 10:58:04.523874 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:04.523810 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b" podStartSLOduration=1.523791705 podStartE2EDuration="1.523791705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:58:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:58:04.523001939 +0000 UTC m=+1822.497084493" watchObservedRunningTime="2026-08-04 10:58:04.523791705 +0000 UTC m=+1822.497874232" Aug 04 10:58:04.542781 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:04.542739 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg" podStartSLOduration=1.5427264219999999 podStartE2EDuration="1.542726422s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:58:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:58:04.541650799 +0000 UTC m=+1822.515733348" watchObservedRunningTime="2026-08-04 10:58:04.542726422 +0000 UTC m=+1822.516808948" Aug 04 10:58:15.514654 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:15.514621 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg" Aug 04 10:58:15.515093 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:15.514799 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b" Aug 04 10:58:43.453460 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:43.453419 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:58:43.522318 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:43.522115 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg"] Aug 04 10:58:43.522599 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:43.522391 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg" podUID="f2e41ff2-e104-4255-88a0-87c86fe2c501" containerName="payload-processing" containerID="cri-o://91c90c5c39c6ed76e357efa69150687d4a31276b0898b2d62283abbfa3651197" gracePeriod=30 Aug 04 10:58:43.528942 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:43.528914 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b"] Aug 04 10:58:43.529147 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:43.529126 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b" podUID="79188947-e71a-4461-aa66-0b3d1997ae13" containerName="payload-pre-processing" containerID="cri-o://a6543c4adff69cc0dde7ae6a0dc7b7680348646a1498efaeff5d1cf721967d95" gracePeriod=30 Aug 04 10:58:43.560063 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:43.559973 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:58:43.657442 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:43.657414 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:58:43.689315 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:43.689280 2535 generic.go:356] "Generic (PLEG): container finished" podID="79188947-e71a-4461-aa66-0b3d1997ae13" containerID="a6543c4adff69cc0dde7ae6a0dc7b7680348646a1498efaeff5d1cf721967d95" exitCode=0 Aug 04 10:58:43.689525 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:43.689357 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b" event={"ID":"79188947-e71a-4461-aa66-0b3d1997ae13","Type":"ContainerDied","Data":"a6543c4adff69cc0dde7ae6a0dc7b7680348646a1498efaeff5d1cf721967d95"} Aug 04 10:58:43.691163 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:43.691134 2535 generic.go:356] "Generic (PLEG): container finished" podID="f2e41ff2-e104-4255-88a0-87c86fe2c501" containerID="91c90c5c39c6ed76e357efa69150687d4a31276b0898b2d62283abbfa3651197" exitCode=0 Aug 04 10:58:43.691310 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:43.691178 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg" event={"ID":"f2e41ff2-e104-4255-88a0-87c86fe2c501","Type":"ContainerDied","Data":"91c90c5c39c6ed76e357efa69150687d4a31276b0898b2d62283abbfa3651197"} Aug 04 10:58:43.760016 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:43.759990 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:58:43.819793 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:43.819768 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg" Aug 04 10:58:43.822580 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:43.822564 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b" Aug 04 10:58:43.848463 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:43.848441 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79188947-e71a-4461-aa66-0b3d1997ae13-plugins-config-volume\") pod \"79188947-e71a-4461-aa66-0b3d1997ae13\" (UID: \"79188947-e71a-4461-aa66-0b3d1997ae13\") " Aug 04 10:58:43.848585 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:43.848543 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lkkm\" (UniqueName: \"kubernetes.io/projected/79188947-e71a-4461-aa66-0b3d1997ae13-kube-api-access-7lkkm\") pod \"79188947-e71a-4461-aa66-0b3d1997ae13\" (UID: \"79188947-e71a-4461-aa66-0b3d1997ae13\") " Aug 04 10:58:43.848585 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:43.848566 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7mz7\" (UniqueName: \"kubernetes.io/projected/f2e41ff2-e104-4255-88a0-87c86fe2c501-kube-api-access-r7mz7\") pod \"f2e41ff2-e104-4255-88a0-87c86fe2c501\" (UID: \"f2e41ff2-e104-4255-88a0-87c86fe2c501\") " Aug 04 10:58:43.848707 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:43.848591 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2e41ff2-e104-4255-88a0-87c86fe2c501-plugins-config-volume\") pod \"f2e41ff2-e104-4255-88a0-87c86fe2c501\" (UID: \"f2e41ff2-e104-4255-88a0-87c86fe2c501\") " Aug 04 10:58:43.848991 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:43.848958 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79188947-e71a-4461-aa66-0b3d1997ae13-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "79188947-e71a-4461-aa66-0b3d1997ae13" (UID: "79188947-e71a-4461-aa66-0b3d1997ae13"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:58:43.849098 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:43.848993 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2e41ff2-e104-4255-88a0-87c86fe2c501-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f2e41ff2-e104-4255-88a0-87c86fe2c501" (UID: "f2e41ff2-e104-4255-88a0-87c86fe2c501"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:58:43.850608 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:43.850587 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79188947-e71a-4461-aa66-0b3d1997ae13-kube-api-access-7lkkm" (OuterVolumeSpecName: "kube-api-access-7lkkm") pod "79188947-e71a-4461-aa66-0b3d1997ae13" (UID: "79188947-e71a-4461-aa66-0b3d1997ae13"). InnerVolumeSpecName "kube-api-access-7lkkm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:58:43.850703 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:43.850678 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2e41ff2-e104-4255-88a0-87c86fe2c501-kube-api-access-r7mz7" (OuterVolumeSpecName: "kube-api-access-r7mz7") pod "f2e41ff2-e104-4255-88a0-87c86fe2c501" (UID: "f2e41ff2-e104-4255-88a0-87c86fe2c501"). InnerVolumeSpecName "kube-api-access-r7mz7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:58:43.949481 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:43.949446 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lkkm\" (UniqueName: \"kubernetes.io/projected/79188947-e71a-4461-aa66-0b3d1997ae13-kube-api-access-7lkkm\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:58:43.949481 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:43.949478 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7mz7\" (UniqueName: \"kubernetes.io/projected/f2e41ff2-e104-4255-88a0-87c86fe2c501-kube-api-access-r7mz7\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:58:43.949701 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:43.949517 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2e41ff2-e104-4255-88a0-87c86fe2c501-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:58:43.949701 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:43.949531 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79188947-e71a-4461-aa66-0b3d1997ae13-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:58:44.697940 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:44.697905 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b" Aug 04 10:58:44.698400 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:44.697912 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b" event={"ID":"79188947-e71a-4461-aa66-0b3d1997ae13","Type":"ContainerDied","Data":"167b4698cc0518b841e1d0aea4b2c51b887a8e0f3003a7541d444f5ebc3704e2"} Aug 04 10:58:44.698400 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:44.698113 2535 scope.go:117] "RemoveContainer" containerID="a6543c4adff69cc0dde7ae6a0dc7b7680348646a1498efaeff5d1cf721967d95" Aug 04 10:58:44.699541 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:44.699514 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg" event={"ID":"f2e41ff2-e104-4255-88a0-87c86fe2c501","Type":"ContainerDied","Data":"7d27c0f66d9dc724baf5165e2d4fbe56bc7ecfde905551203d98ea87869c018e"} Aug 04 10:58:44.699723 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:44.699539 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg" Aug 04 10:58:44.707851 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:44.707835 2535 scope.go:117] "RemoveContainer" containerID="91c90c5c39c6ed76e357efa69150687d4a31276b0898b2d62283abbfa3651197" Aug 04 10:58:44.718803 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:44.718781 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b"] Aug 04 10:58:44.722895 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:44.722875 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fb58dd84-c48dfbf76-pr76b"] Aug 04 10:58:44.732521 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:44.732499 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg"] Aug 04 10:58:44.735988 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:44.735969 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fb58dd84-7898fbbdcd-jntgg"] Aug 04 10:58:44.918532 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:44.918481 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:58:45.183304 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:45.183216 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:58:45.403154 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:45.403116 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:58:45.743294 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:45.743262 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:58:45.906059 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:45.906025 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:58:46.480165 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:46.480134 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79188947-e71a-4461-aa66-0b3d1997ae13" path="/var/lib/kubelet/pods/79188947-e71a-4461-aa66-0b3d1997ae13/volumes" Aug 04 10:58:46.480499 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:46.480473 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2e41ff2-e104-4255-88a0-87c86fe2c501" path="/var/lib/kubelet/pods/f2e41ff2-e104-4255-88a0-87c86fe2c501/volumes" Aug 04 10:58:47.887159 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:47.887125 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:58:47.985888 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:47.985856 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:58:49.167763 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:49.167720 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:58:49.584547 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:49.584517 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:58:49.697018 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:49.696988 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:58:49.807855 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:49.807826 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:58:49.890548 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:49.890433 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4"] Aug 04 10:58:49.890896 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:49.890876 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2e41ff2-e104-4255-88a0-87c86fe2c501" containerName="payload-processing" Aug 04 10:58:49.890983 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:49.890897 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2e41ff2-e104-4255-88a0-87c86fe2c501" containerName="payload-processing" Aug 04 10:58:49.890983 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:49.890917 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79188947-e71a-4461-aa66-0b3d1997ae13" containerName="payload-pre-processing" Aug 04 10:58:49.890983 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:49.890925 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="79188947-e71a-4461-aa66-0b3d1997ae13" containerName="payload-pre-processing" Aug 04 10:58:49.891130 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:49.891053 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="79188947-e71a-4461-aa66-0b3d1997ae13" containerName="payload-pre-processing" Aug 04 10:58:49.891130 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:49.891070 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2e41ff2-e104-4255-88a0-87c86fe2c501" containerName="payload-processing" Aug 04 10:58:49.894534 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:49.894514 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4" Aug 04 10:58:49.896899 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:49.896880 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-524f2cac\"" Aug 04 10:58:49.897081 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:49.897063 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-524f2cac-dockercfg-z94px\"" Aug 04 10:58:49.905662 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:49.905643 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4"] Aug 04 10:58:49.963915 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:49.963879 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9"] Aug 04 10:58:49.967298 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:49.967278 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9" Aug 04 10:58:49.980012 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:49.979985 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9"] Aug 04 10:58:50.004571 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:50.004542 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzq8h\" (UniqueName: \"kubernetes.io/projected/4c2f0a95-bfcd-4004-b1c7-be48ad97ead9-kube-api-access-bzq8h\") pod \"payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4\" (UID: \"4c2f0a95-bfcd-4004-b1c7-be48ad97ead9\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4" Aug 04 10:58:50.004700 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:50.004579 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c2f0a95-bfcd-4004-b1c7-be48ad97ead9-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4\" (UID: \"4c2f0a95-bfcd-4004-b1c7-be48ad97ead9\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4" Aug 04 10:58:50.105868 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:50.105822 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzq8h\" (UniqueName: \"kubernetes.io/projected/4c2f0a95-bfcd-4004-b1c7-be48ad97ead9-kube-api-access-bzq8h\") pod \"payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4\" (UID: \"4c2f0a95-bfcd-4004-b1c7-be48ad97ead9\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4" Aug 04 10:58:50.105868 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:50.105861 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c2f0a95-bfcd-4004-b1c7-be48ad97ead9-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4\" (UID: \"4c2f0a95-bfcd-4004-b1c7-be48ad97ead9\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4" Aug 04 10:58:50.106056 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:50.105890 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzlsh\" (UniqueName: \"kubernetes.io/projected/ca4ba96f-c0b3-473e-bb10-7ce073130048-kube-api-access-xzlsh\") pod \"payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9\" (UID: \"ca4ba96f-c0b3-473e-bb10-7ce073130048\") " pod="openshift-ingress/payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9" Aug 04 10:58:50.106056 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:50.105942 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca4ba96f-c0b3-473e-bb10-7ce073130048-plugins-config-volume\") pod \"payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9\" (UID: \"ca4ba96f-c0b3-473e-bb10-7ce073130048\") " pod="openshift-ingress/payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9" Aug 04 10:58:50.106834 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:50.106811 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c2f0a95-bfcd-4004-b1c7-be48ad97ead9-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4\" (UID: \"4c2f0a95-bfcd-4004-b1c7-be48ad97ead9\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4" Aug 04 10:58:50.114295 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:50.114272 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzq8h\" (UniqueName: \"kubernetes.io/projected/4c2f0a95-bfcd-4004-b1c7-be48ad97ead9-kube-api-access-bzq8h\") pod \"payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4\" (UID: \"4c2f0a95-bfcd-4004-b1c7-be48ad97ead9\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4" Aug 04 10:58:50.205145 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:50.205066 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4" Aug 04 10:58:50.206413 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:50.206380 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzlsh\" (UniqueName: \"kubernetes.io/projected/ca4ba96f-c0b3-473e-bb10-7ce073130048-kube-api-access-xzlsh\") pod \"payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9\" (UID: \"ca4ba96f-c0b3-473e-bb10-7ce073130048\") " pod="openshift-ingress/payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9" Aug 04 10:58:50.206566 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:50.206441 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca4ba96f-c0b3-473e-bb10-7ce073130048-plugins-config-volume\") pod \"payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9\" (UID: \"ca4ba96f-c0b3-473e-bb10-7ce073130048\") " pod="openshift-ingress/payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9" Aug 04 10:58:50.207064 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:50.207045 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca4ba96f-c0b3-473e-bb10-7ce073130048-plugins-config-volume\") pod \"payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9\" (UID: \"ca4ba96f-c0b3-473e-bb10-7ce073130048\") " pod="openshift-ingress/payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9" Aug 04 10:58:50.214333 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:50.214311 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzlsh\" (UniqueName: \"kubernetes.io/projected/ca4ba96f-c0b3-473e-bb10-7ce073130048-kube-api-access-xzlsh\") pod \"payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9\" (UID: \"ca4ba96f-c0b3-473e-bb10-7ce073130048\") " pod="openshift-ingress/payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9" Aug 04 10:58:50.277267 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:50.277239 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9" Aug 04 10:58:50.340050 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:50.340017 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4"] Aug 04 10:58:50.344584 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:58:50.344559 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c2f0a95_bfcd_4004_b1c7_be48ad97ead9.slice/crio-0b8c13295f4b8a03fa8670806f9d37408571e85ee1efc3e7394070d06649bab4 WatchSource:0}: Error finding container 0b8c13295f4b8a03fa8670806f9d37408571e85ee1efc3e7394070d06649bab4: Status 404 returned error can't find the container with id 0b8c13295f4b8a03fa8670806f9d37408571e85ee1efc3e7394070d06649bab4 Aug 04 10:58:50.634122 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:50.634098 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9"] Aug 04 10:58:50.636603 ip-10-0-130-250 kubenswrapper[2535]: W0804 10:58:50.636572 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca4ba96f_c0b3_473e_bb10_7ce073130048.slice/crio-98b1a49467f0c398fb9332f81d8b52cceda44d3483946f912006017e005375ed WatchSource:0}: Error finding container 98b1a49467f0c398fb9332f81d8b52cceda44d3483946f912006017e005375ed: Status 404 returned error can't find the container with id 98b1a49467f0c398fb9332f81d8b52cceda44d3483946f912006017e005375ed Aug 04 10:58:50.733555 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:50.733518 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9" event={"ID":"ca4ba96f-c0b3-473e-bb10-7ce073130048","Type":"ContainerStarted","Data":"c05bb9515e16c3073c61f6f9f834c89a4ec8536402b665ef857413e000f84dc3"} Aug 04 10:58:50.733731 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:50.733595 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9" event={"ID":"ca4ba96f-c0b3-473e-bb10-7ce073130048","Type":"ContainerStarted","Data":"98b1a49467f0c398fb9332f81d8b52cceda44d3483946f912006017e005375ed"} Aug 04 10:58:50.733731 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:50.733657 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9" Aug 04 10:58:50.735336 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:50.735310 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4" event={"ID":"4c2f0a95-bfcd-4004-b1c7-be48ad97ead9","Type":"ContainerStarted","Data":"c1829fe1ff4e768bd93af89bdf956c7452272613b2ad3680d0f7bd98c87be7da"} Aug 04 10:58:50.735481 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:50.735353 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4" event={"ID":"4c2f0a95-bfcd-4004-b1c7-be48ad97ead9","Type":"ContainerStarted","Data":"0b8c13295f4b8a03fa8670806f9d37408571e85ee1efc3e7394070d06649bab4"} Aug 04 10:58:50.735481 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:50.735465 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4" Aug 04 10:58:50.752613 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:50.752559 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9" podStartSLOduration=1.752543589 podStartE2EDuration="1.752543589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:58:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:58:50.750461786 +0000 UTC m=+1868.724544313" watchObservedRunningTime="2026-08-04 10:58:50.752543589 +0000 UTC m=+1868.726626115" Aug 04 10:58:50.771583 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:58:50.771530 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4" podStartSLOduration=1.771515349 podStartE2EDuration="1.771515349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:58:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:58:50.769635659 +0000 UTC m=+1868.743718206" watchObservedRunningTime="2026-08-04 10:58:50.771515349 +0000 UTC m=+1868.745597877" Aug 04 10:59:01.742060 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:01.742031 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4" Aug 04 10:59:01.742478 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:01.742323 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9" Aug 04 10:59:19.592765 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:19.592652 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:59:19.615756 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:19.615723 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9"] Aug 04 10:59:19.615995 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:19.615966 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9" podUID="ca4ba96f-c0b3-473e-bb10-7ce073130048" containerName="payload-processing" containerID="cri-o://c05bb9515e16c3073c61f6f9f834c89a4ec8536402b665ef857413e000f84dc3" gracePeriod=30 Aug 04 10:59:19.623222 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:19.622160 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4"] Aug 04 10:59:19.623222 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:19.622376 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4" podUID="4c2f0a95-bfcd-4004-b1c7-be48ad97ead9" containerName="payload-pre-processing" containerID="cri-o://c1829fe1ff4e768bd93af89bdf956c7452272613b2ad3680d0f7bd98c87be7da" gracePeriod=30 Aug 04 10:59:19.666930 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:19.666904 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:59:19.767210 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:19.767136 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:59:19.841326 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:19.841297 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:59:19.872291 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:19.872214 2535 generic.go:356] "Generic (PLEG): container finished" podID="ca4ba96f-c0b3-473e-bb10-7ce073130048" containerID="c05bb9515e16c3073c61f6f9f834c89a4ec8536402b665ef857413e000f84dc3" exitCode=0 Aug 04 10:59:19.872291 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:19.872282 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9" event={"ID":"ca4ba96f-c0b3-473e-bb10-7ce073130048","Type":"ContainerDied","Data":"c05bb9515e16c3073c61f6f9f834c89a4ec8536402b665ef857413e000f84dc3"} Aug 04 10:59:19.873964 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:19.873938 2535 generic.go:356] "Generic (PLEG): container finished" podID="4c2f0a95-bfcd-4004-b1c7-be48ad97ead9" containerID="c1829fe1ff4e768bd93af89bdf956c7452272613b2ad3680d0f7bd98c87be7da" exitCode=0 Aug 04 10:59:19.874100 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:19.873990 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4" event={"ID":"4c2f0a95-bfcd-4004-b1c7-be48ad97ead9","Type":"ContainerDied","Data":"c1829fe1ff4e768bd93af89bdf956c7452272613b2ad3680d0f7bd98c87be7da"} Aug 04 10:59:19.913882 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:19.913860 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9" Aug 04 10:59:19.914953 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:19.914936 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4" Aug 04 10:59:19.923146 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:19.923091 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:59:20.064277 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:20.064243 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzq8h\" (UniqueName: \"kubernetes.io/projected/4c2f0a95-bfcd-4004-b1c7-be48ad97ead9-kube-api-access-bzq8h\") pod \"4c2f0a95-bfcd-4004-b1c7-be48ad97ead9\" (UID: \"4c2f0a95-bfcd-4004-b1c7-be48ad97ead9\") " Aug 04 10:59:20.064444 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:20.064337 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c2f0a95-bfcd-4004-b1c7-be48ad97ead9-plugins-config-volume\") pod \"4c2f0a95-bfcd-4004-b1c7-be48ad97ead9\" (UID: \"4c2f0a95-bfcd-4004-b1c7-be48ad97ead9\") " Aug 04 10:59:20.064444 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:20.064356 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzlsh\" (UniqueName: \"kubernetes.io/projected/ca4ba96f-c0b3-473e-bb10-7ce073130048-kube-api-access-xzlsh\") pod \"ca4ba96f-c0b3-473e-bb10-7ce073130048\" (UID: \"ca4ba96f-c0b3-473e-bb10-7ce073130048\") " Aug 04 10:59:20.064444 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:20.064387 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca4ba96f-c0b3-473e-bb10-7ce073130048-plugins-config-volume\") pod \"ca4ba96f-c0b3-473e-bb10-7ce073130048\" (UID: \"ca4ba96f-c0b3-473e-bb10-7ce073130048\") " Aug 04 10:59:20.064735 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:20.064712 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c2f0a95-bfcd-4004-b1c7-be48ad97ead9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4c2f0a95-bfcd-4004-b1c7-be48ad97ead9" (UID: "4c2f0a95-bfcd-4004-b1c7-be48ad97ead9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:59:20.064806 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:20.064786 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca4ba96f-c0b3-473e-bb10-7ce073130048-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ca4ba96f-c0b3-473e-bb10-7ce073130048" (UID: "ca4ba96f-c0b3-473e-bb10-7ce073130048"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:59:20.066439 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:20.066420 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca4ba96f-c0b3-473e-bb10-7ce073130048-kube-api-access-xzlsh" (OuterVolumeSpecName: "kube-api-access-xzlsh") pod "ca4ba96f-c0b3-473e-bb10-7ce073130048" (UID: "ca4ba96f-c0b3-473e-bb10-7ce073130048"). InnerVolumeSpecName "kube-api-access-xzlsh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:59:20.066439 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:20.066427 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c2f0a95-bfcd-4004-b1c7-be48ad97ead9-kube-api-access-bzq8h" (OuterVolumeSpecName: "kube-api-access-bzq8h") pod "4c2f0a95-bfcd-4004-b1c7-be48ad97ead9" (UID: "4c2f0a95-bfcd-4004-b1c7-be48ad97ead9"). InnerVolumeSpecName "kube-api-access-bzq8h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:59:20.165945 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:20.165866 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzlsh\" (UniqueName: \"kubernetes.io/projected/ca4ba96f-c0b3-473e-bb10-7ce073130048-kube-api-access-xzlsh\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:59:20.165945 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:20.165890 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca4ba96f-c0b3-473e-bb10-7ce073130048-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:59:20.165945 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:20.165900 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzq8h\" (UniqueName: \"kubernetes.io/projected/4c2f0a95-bfcd-4004-b1c7-be48ad97ead9-kube-api-access-bzq8h\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:59:20.165945 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:20.165911 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c2f0a95-bfcd-4004-b1c7-be48ad97ead9-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 10:59:20.179215 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:20.179188 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:59:20.696640 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:20.696604 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:59:20.797178 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:20.797139 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:59:20.879782 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:20.879755 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9" Aug 04 10:59:20.879782 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:20.879759 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9" event={"ID":"ca4ba96f-c0b3-473e-bb10-7ce073130048","Type":"ContainerDied","Data":"98b1a49467f0c398fb9332f81d8b52cceda44d3483946f912006017e005375ed"} Aug 04 10:59:20.880008 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:20.879871 2535 scope.go:117] "RemoveContainer" containerID="c05bb9515e16c3073c61f6f9f834c89a4ec8536402b665ef857413e000f84dc3" Aug 04 10:59:20.881329 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:20.881306 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4" event={"ID":"4c2f0a95-bfcd-4004-b1c7-be48ad97ead9","Type":"ContainerDied","Data":"0b8c13295f4b8a03fa8670806f9d37408571e85ee1efc3e7394070d06649bab4"} Aug 04 10:59:20.881329 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:20.881315 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4" Aug 04 10:59:20.889338 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:20.889322 2535 scope.go:117] "RemoveContainer" containerID="c1829fe1ff4e768bd93af89bdf956c7452272613b2ad3680d0f7bd98c87be7da" Aug 04 10:59:20.905533 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:20.905450 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:59:20.906694 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:20.906671 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9"] Aug 04 10:59:20.908718 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:20.908701 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-524f2cac-dc9ccbd5d-zr5f9"] Aug 04 10:59:20.919851 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:20.919833 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4"] Aug 04 10:59:20.925168 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:20.925149 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-524f2cac-7499dc656-vszj4"] Aug 04 10:59:22.480993 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:22.480964 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c2f0a95-bfcd-4004-b1c7-be48ad97ead9" path="/var/lib/kubelet/pods/4c2f0a95-bfcd-4004-b1c7-be48ad97ead9/volumes" Aug 04 10:59:22.481347 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:22.481309 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca4ba96f-c0b3-473e-bb10-7ce073130048" path="/var/lib/kubelet/pods/ca4ba96f-c0b3-473e-bb10-7ce073130048/volumes" Aug 04 10:59:32.677187 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:32.677153 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:59:40.739499 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:40.739460 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:59:40.905279 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:40.905240 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:59:41.059481 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:41.059425 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 10:59:41.552101 ip-10-0-130-250 kubenswrapper[2535]: I0804 10:59:41.552056 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:00:00.142582 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:00.142548 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764020-28nbf"] Aug 04 11:00:00.142990 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:00.142888 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c2f0a95-bfcd-4004-b1c7-be48ad97ead9" containerName="payload-pre-processing" Aug 04 11:00:00.142990 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:00.142899 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c2f0a95-bfcd-4004-b1c7-be48ad97ead9" containerName="payload-pre-processing" Aug 04 11:00:00.142990 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:00.142910 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca4ba96f-c0b3-473e-bb10-7ce073130048" containerName="payload-processing" Aug 04 11:00:00.142990 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:00.142915 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca4ba96f-c0b3-473e-bb10-7ce073130048" containerName="payload-processing" Aug 04 11:00:00.142990 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:00.142976 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca4ba96f-c0b3-473e-bb10-7ce073130048" containerName="payload-processing" Aug 04 11:00:00.142990 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:00.142984 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c2f0a95-bfcd-4004-b1c7-be48ad97ead9" containerName="payload-pre-processing" Aug 04 11:00:00.145926 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:00.145912 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764020-28nbf" Aug 04 11:00:00.148154 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:00.148127 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qptbt\"" Aug 04 11:00:00.153826 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:00.153807 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764020-28nbf"] Aug 04 11:00:00.155982 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:00.155963 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764020-28nbf" Aug 04 11:00:00.289394 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:00.289295 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764020-28nbf"] Aug 04 11:00:00.292213 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:00:00.292184 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac1f65a7_7177_4828_8a79_5e79e204c3a5.slice/crio-203b3a51469bada3dee7ba1eb3a813549b49ffb99ff53d9b0648b5ddca1b4e68 WatchSource:0}: Error finding container 203b3a51469bada3dee7ba1eb3a813549b49ffb99ff53d9b0648b5ddca1b4e68: Status 404 returned error can't find the container with id 203b3a51469bada3dee7ba1eb3a813549b49ffb99ff53d9b0648b5ddca1b4e68 Aug 04 11:00:01.062066 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:01.062028 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764020-28nbf" event={"ID":"ac1f65a7-7177-4828-8a79-5e79e204c3a5","Type":"ContainerStarted","Data":"9204e2d64f388161170c2ba0b17e3bac4c8e13f1334f2581ab2abd1b398af541"} Aug 04 11:00:01.062262 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:01.062111 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764020-28nbf" event={"ID":"ac1f65a7-7177-4828-8a79-5e79e204c3a5","Type":"ContainerStarted","Data":"203b3a51469bada3dee7ba1eb3a813549b49ffb99ff53d9b0648b5ddca1b4e68"} Aug 04 11:00:01.079591 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:01.079543 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764020-28nbf" podStartSLOduration=1.07952886 podStartE2EDuration="1.07952886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:00:01.078443296 +0000 UTC m=+1939.052525847" watchObservedRunningTime="2026-08-04 11:00:01.07952886 +0000 UTC m=+1939.053611386" Aug 04 11:00:02.068796 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:02.068760 2535 generic.go:356] "Generic (PLEG): container finished" podID="ac1f65a7-7177-4828-8a79-5e79e204c3a5" containerID="9204e2d64f388161170c2ba0b17e3bac4c8e13f1334f2581ab2abd1b398af541" exitCode=0 Aug 04 11:00:02.069172 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:02.068828 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764020-28nbf" event={"ID":"ac1f65a7-7177-4828-8a79-5e79e204c3a5","Type":"ContainerDied","Data":"9204e2d64f388161170c2ba0b17e3bac4c8e13f1334f2581ab2abd1b398af541"} Aug 04 11:00:03.228204 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:03.228177 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764020-28nbf" Aug 04 11:00:04.080206 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:04.080176 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764020-28nbf" Aug 04 11:00:04.080206 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:04.080196 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764020-28nbf" event={"ID":"ac1f65a7-7177-4828-8a79-5e79e204c3a5","Type":"ContainerDied","Data":"203b3a51469bada3dee7ba1eb3a813549b49ffb99ff53d9b0648b5ddca1b4e68"} Aug 04 11:00:04.080473 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:04.080238 2535 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="203b3a51469bada3dee7ba1eb3a813549b49ffb99ff53d9b0648b5ddca1b4e68" Aug 04 11:00:07.592567 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:07.592532 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:00:14.106910 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:14.106872 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:00:14.211059 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:14.211024 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:00:14.309793 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:14.309759 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:00:30.255393 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:30.255356 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:00:36.293780 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:36.293695 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:00:36.398906 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:36.398873 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:00:36.495876 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:36.495846 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:00:41.908686 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:41.908634 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:00:47.061464 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:47.061420 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:00:58.160412 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:00:58.160379 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:01:05.002514 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:05.002452 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:01:05.094395 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:05.094361 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:01:05.526205 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:05.526164 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:01:05.625227 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:05.625193 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:01:05.992138 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:05.992047 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:01:09.530367 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:09.530334 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:01:13.778842 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:13.778808 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:01:13.880034 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:13.880001 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:01:14.454096 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:14.454063 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:01:21.776295 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:21.776261 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:01:24.614920 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:24.614885 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:01:24.716133 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:24.716098 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:01:25.242358 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:25.242316 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:01:25.341934 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:25.341903 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:01:26.748776 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:26.748744 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:01:27.013721 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:27.013648 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:01:27.378133 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:27.378092 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:01:27.476206 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:27.476159 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:01:27.484436 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:27.484408 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn"] Aug 04 11:01:27.484972 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:27.484945 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac1f65a7-7177-4828-8a79-5e79e204c3a5" containerName="cleanup" Aug 04 11:01:27.484972 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:27.484966 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac1f65a7-7177-4828-8a79-5e79e204c3a5" containerName="cleanup" Aug 04 11:01:27.485149 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:27.485039 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac1f65a7-7177-4828-8a79-5e79e204c3a5" containerName="cleanup" Aug 04 11:01:27.487941 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:27.487927 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn" Aug 04 11:01:27.489845 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:27.489826 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-fe15eb98\"" Aug 04 11:01:27.489941 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:27.489925 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-fe15eb98-dockercfg-v4vtl\"" Aug 04 11:01:27.496518 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:27.496456 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv"] Aug 04 11:01:27.500305 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:27.500285 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn"] Aug 04 11:01:27.500404 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:27.500365 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv" Aug 04 11:01:27.511243 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:27.511223 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv"] Aug 04 11:01:27.569573 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:27.569541 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hkhx\" (UniqueName: \"kubernetes.io/projected/6fd57c23-d075-4661-a033-dcbbd537c291-kube-api-access-8hkhx\") pod \"payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn\" (UID: \"6fd57c23-d075-4661-a033-dcbbd537c291\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn" Aug 04 11:01:27.569573 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:27.569575 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a431d341-30e7-4cf1-a87f-93c0536c1343-plugins-config-volume\") pod \"payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv\" (UID: \"a431d341-30e7-4cf1-a87f-93c0536c1343\") " pod="openshift-ingress/payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv" Aug 04 11:01:27.569749 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:27.569679 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7m65g\" (UniqueName: \"kubernetes.io/projected/a431d341-30e7-4cf1-a87f-93c0536c1343-kube-api-access-7m65g\") pod \"payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv\" (UID: \"a431d341-30e7-4cf1-a87f-93c0536c1343\") " pod="openshift-ingress/payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv" Aug 04 11:01:27.569749 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:27.569733 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fd57c23-d075-4661-a033-dcbbd537c291-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn\" (UID: \"6fd57c23-d075-4661-a033-dcbbd537c291\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn" Aug 04 11:01:27.670393 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:27.670320 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7m65g\" (UniqueName: \"kubernetes.io/projected/a431d341-30e7-4cf1-a87f-93c0536c1343-kube-api-access-7m65g\") pod \"payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv\" (UID: \"a431d341-30e7-4cf1-a87f-93c0536c1343\") " pod="openshift-ingress/payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv" Aug 04 11:01:27.670393 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:27.670370 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fd57c23-d075-4661-a033-dcbbd537c291-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn\" (UID: \"6fd57c23-d075-4661-a033-dcbbd537c291\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn" Aug 04 11:01:27.670642 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:27.670399 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hkhx\" (UniqueName: \"kubernetes.io/projected/6fd57c23-d075-4661-a033-dcbbd537c291-kube-api-access-8hkhx\") pod \"payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn\" (UID: \"6fd57c23-d075-4661-a033-dcbbd537c291\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn" Aug 04 11:01:27.670642 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:27.670422 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a431d341-30e7-4cf1-a87f-93c0536c1343-plugins-config-volume\") pod \"payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv\" (UID: \"a431d341-30e7-4cf1-a87f-93c0536c1343\") " pod="openshift-ingress/payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv" Aug 04 11:01:27.671122 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:27.671101 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a431d341-30e7-4cf1-a87f-93c0536c1343-plugins-config-volume\") pod \"payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv\" (UID: \"a431d341-30e7-4cf1-a87f-93c0536c1343\") " pod="openshift-ingress/payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv" Aug 04 11:01:27.671179 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:27.671135 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fd57c23-d075-4661-a033-dcbbd537c291-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn\" (UID: \"6fd57c23-d075-4661-a033-dcbbd537c291\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn" Aug 04 11:01:27.678494 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:27.678453 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hkhx\" (UniqueName: \"kubernetes.io/projected/6fd57c23-d075-4661-a033-dcbbd537c291-kube-api-access-8hkhx\") pod \"payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn\" (UID: \"6fd57c23-d075-4661-a033-dcbbd537c291\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn" Aug 04 11:01:27.678611 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:27.678521 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7m65g\" (UniqueName: \"kubernetes.io/projected/a431d341-30e7-4cf1-a87f-93c0536c1343-kube-api-access-7m65g\") pod \"payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv\" (UID: \"a431d341-30e7-4cf1-a87f-93c0536c1343\") " pod="openshift-ingress/payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv" Aug 04 11:01:27.799785 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:27.799754 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn" Aug 04 11:01:27.812993 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:27.812969 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv" Aug 04 11:01:27.967602 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:27.967576 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv"] Aug 04 11:01:27.968944 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:01:27.968922 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda431d341_30e7_4cf1_a87f_93c0536c1343.slice/crio-6fb8ce5a9b44fc1a0b2724a7dde36776b19fcc12a8ee87ae046aedb8bf1a9f3a WatchSource:0}: Error finding container 6fb8ce5a9b44fc1a0b2724a7dde36776b19fcc12a8ee87ae046aedb8bf1a9f3a: Status 404 returned error can't find the container with id 6fb8ce5a9b44fc1a0b2724a7dde36776b19fcc12a8ee87ae046aedb8bf1a9f3a Aug 04 11:01:28.146196 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:28.146156 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn"] Aug 04 11:01:28.148322 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:01:28.148301 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fd57c23_d075_4661_a033_dcbbd537c291.slice/crio-07a97a204709dab9ae27e14e92a4d262afdf92d40188228c378d98ccf766824e WatchSource:0}: Error finding container 07a97a204709dab9ae27e14e92a4d262afdf92d40188228c378d98ccf766824e: Status 404 returned error can't find the container with id 07a97a204709dab9ae27e14e92a4d262afdf92d40188228c378d98ccf766824e Aug 04 11:01:28.461341 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:28.461310 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn" event={"ID":"6fd57c23-d075-4661-a033-dcbbd537c291","Type":"ContainerStarted","Data":"a2ae77257100a13257dfd72683c6fd5ab766ccc5ca4dff9c9dddee1f773bc00d"} Aug 04 11:01:28.461543 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:28.461358 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn" event={"ID":"6fd57c23-d075-4661-a033-dcbbd537c291","Type":"ContainerStarted","Data":"07a97a204709dab9ae27e14e92a4d262afdf92d40188228c378d98ccf766824e"} Aug 04 11:01:28.461543 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:28.461401 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn" Aug 04 11:01:28.463039 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:28.463012 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv" event={"ID":"a431d341-30e7-4cf1-a87f-93c0536c1343","Type":"ContainerStarted","Data":"b4095be759cff7d8c659f27c0f7ebcd74a8c400cb84871a7380b747231c86aea"} Aug 04 11:01:28.463174 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:28.463057 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv" event={"ID":"a431d341-30e7-4cf1-a87f-93c0536c1343","Type":"ContainerStarted","Data":"6fb8ce5a9b44fc1a0b2724a7dde36776b19fcc12a8ee87ae046aedb8bf1a9f3a"} Aug 04 11:01:28.463174 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:28.463166 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv" Aug 04 11:01:28.479507 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:28.479242 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn" podStartSLOduration=1.479223008 podStartE2EDuration="1.479223008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:01:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:01:28.477878025 +0000 UTC m=+2026.451960592" watchObservedRunningTime="2026-08-04 11:01:28.479223008 +0000 UTC m=+2026.453305536" Aug 04 11:01:28.494109 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:28.494061 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv" podStartSLOduration=1.494046698 podStartE2EDuration="1.494046698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:01:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:01:28.491897334 +0000 UTC m=+2026.465979861" watchObservedRunningTime="2026-08-04 11:01:28.494046698 +0000 UTC m=+2026.468129225" Aug 04 11:01:39.471046 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:39.471010 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv" Aug 04 11:01:39.471616 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:39.471410 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn" Aug 04 11:01:40.032752 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:40.032717 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:01:40.741254 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:40.741217 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:01:40.834305 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:40.834276 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:01:41.183711 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:41.183671 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:01:47.969716 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:47.969677 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:01:48.069312 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:48.069278 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:01:51.981699 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:51.981665 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:01:53.809094 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:53.809060 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv"] Aug 04 11:01:53.809601 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:53.809272 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv" podUID="a431d341-30e7-4cf1-a87f-93c0536c1343" containerName="payload-processing" containerID="cri-o://b4095be759cff7d8c659f27c0f7ebcd74a8c400cb84871a7380b747231c86aea" gracePeriod=30 Aug 04 11:01:53.816020 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:53.815990 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn"] Aug 04 11:01:53.816249 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:53.816204 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn" podUID="6fd57c23-d075-4661-a033-dcbbd537c291" containerName="payload-pre-processing" containerID="cri-o://a2ae77257100a13257dfd72683c6fd5ab766ccc5ca4dff9c9dddee1f773bc00d" gracePeriod=30 Aug 04 11:01:54.069996 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.069931 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv" Aug 04 11:01:54.098870 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.098838 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn" Aug 04 11:01:54.113254 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.113219 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7m65g\" (UniqueName: \"kubernetes.io/projected/a431d341-30e7-4cf1-a87f-93c0536c1343-kube-api-access-7m65g\") pod \"a431d341-30e7-4cf1-a87f-93c0536c1343\" (UID: \"a431d341-30e7-4cf1-a87f-93c0536c1343\") " Aug 04 11:01:54.113406 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.113265 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fd57c23-d075-4661-a033-dcbbd537c291-plugins-config-volume\") pod \"6fd57c23-d075-4661-a033-dcbbd537c291\" (UID: \"6fd57c23-d075-4661-a033-dcbbd537c291\") " Aug 04 11:01:54.113406 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.113310 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hkhx\" (UniqueName: \"kubernetes.io/projected/6fd57c23-d075-4661-a033-dcbbd537c291-kube-api-access-8hkhx\") pod \"6fd57c23-d075-4661-a033-dcbbd537c291\" (UID: \"6fd57c23-d075-4661-a033-dcbbd537c291\") " Aug 04 11:01:54.113406 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.113326 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a431d341-30e7-4cf1-a87f-93c0536c1343-plugins-config-volume\") pod \"a431d341-30e7-4cf1-a87f-93c0536c1343\" (UID: \"a431d341-30e7-4cf1-a87f-93c0536c1343\") " Aug 04 11:01:54.113742 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.113711 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fd57c23-d075-4661-a033-dcbbd537c291-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6fd57c23-d075-4661-a033-dcbbd537c291" (UID: "6fd57c23-d075-4661-a033-dcbbd537c291"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:01:54.113819 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.113762 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a431d341-30e7-4cf1-a87f-93c0536c1343-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a431d341-30e7-4cf1-a87f-93c0536c1343" (UID: "a431d341-30e7-4cf1-a87f-93c0536c1343"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:01:54.116347 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.116283 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fd57c23-d075-4661-a033-dcbbd537c291-kube-api-access-8hkhx" (OuterVolumeSpecName: "kube-api-access-8hkhx") pod "6fd57c23-d075-4661-a033-dcbbd537c291" (UID: "6fd57c23-d075-4661-a033-dcbbd537c291"). InnerVolumeSpecName "kube-api-access-8hkhx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 11:01:54.116347 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.116296 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a431d341-30e7-4cf1-a87f-93c0536c1343-kube-api-access-7m65g" (OuterVolumeSpecName: "kube-api-access-7m65g") pod "a431d341-30e7-4cf1-a87f-93c0536c1343" (UID: "a431d341-30e7-4cf1-a87f-93c0536c1343"). InnerVolumeSpecName "kube-api-access-7m65g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 11:01:54.214528 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.214482 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7m65g\" (UniqueName: \"kubernetes.io/projected/a431d341-30e7-4cf1-a87f-93c0536c1343-kube-api-access-7m65g\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:01:54.214528 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.214528 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fd57c23-d075-4661-a033-dcbbd537c291-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:01:54.214762 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.214542 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hkhx\" (UniqueName: \"kubernetes.io/projected/6fd57c23-d075-4661-a033-dcbbd537c291-kube-api-access-8hkhx\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:01:54.214762 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.214557 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a431d341-30e7-4cf1-a87f-93c0536c1343-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:01:54.590070 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.590036 2535 generic.go:356] "Generic (PLEG): container finished" podID="6fd57c23-d075-4661-a033-dcbbd537c291" containerID="a2ae77257100a13257dfd72683c6fd5ab766ccc5ca4dff9c9dddee1f773bc00d" exitCode=0 Aug 04 11:01:54.590240 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.590080 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn" Aug 04 11:01:54.590240 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.590117 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn" event={"ID":"6fd57c23-d075-4661-a033-dcbbd537c291","Type":"ContainerDied","Data":"a2ae77257100a13257dfd72683c6fd5ab766ccc5ca4dff9c9dddee1f773bc00d"} Aug 04 11:01:54.590240 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.590167 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn" event={"ID":"6fd57c23-d075-4661-a033-dcbbd537c291","Type":"ContainerDied","Data":"07a97a204709dab9ae27e14e92a4d262afdf92d40188228c378d98ccf766824e"} Aug 04 11:01:54.590240 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.590190 2535 scope.go:117] "RemoveContainer" containerID="a2ae77257100a13257dfd72683c6fd5ab766ccc5ca4dff9c9dddee1f773bc00d" Aug 04 11:01:54.591524 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.591503 2535 generic.go:356] "Generic (PLEG): container finished" podID="a431d341-30e7-4cf1-a87f-93c0536c1343" containerID="b4095be759cff7d8c659f27c0f7ebcd74a8c400cb84871a7380b747231c86aea" exitCode=0 Aug 04 11:01:54.591624 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.591558 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv" Aug 04 11:01:54.591624 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.591574 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv" event={"ID":"a431d341-30e7-4cf1-a87f-93c0536c1343","Type":"ContainerDied","Data":"b4095be759cff7d8c659f27c0f7ebcd74a8c400cb84871a7380b747231c86aea"} Aug 04 11:01:54.591624 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.591615 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv" event={"ID":"a431d341-30e7-4cf1-a87f-93c0536c1343","Type":"ContainerDied","Data":"6fb8ce5a9b44fc1a0b2724a7dde36776b19fcc12a8ee87ae046aedb8bf1a9f3a"} Aug 04 11:01:54.599268 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.599251 2535 scope.go:117] "RemoveContainer" containerID="a2ae77257100a13257dfd72683c6fd5ab766ccc5ca4dff9c9dddee1f773bc00d" Aug 04 11:01:54.599563 ip-10-0-130-250 kubenswrapper[2535]: E0804 11:01:54.599545 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2ae77257100a13257dfd72683c6fd5ab766ccc5ca4dff9c9dddee1f773bc00d\": container with ID starting with a2ae77257100a13257dfd72683c6fd5ab766ccc5ca4dff9c9dddee1f773bc00d not found: ID does not exist" containerID="a2ae77257100a13257dfd72683c6fd5ab766ccc5ca4dff9c9dddee1f773bc00d" Aug 04 11:01:54.599632 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.599570 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2ae77257100a13257dfd72683c6fd5ab766ccc5ca4dff9c9dddee1f773bc00d"} err="failed to get container status \"a2ae77257100a13257dfd72683c6fd5ab766ccc5ca4dff9c9dddee1f773bc00d\": rpc error: code = NotFound desc = could not find container \"a2ae77257100a13257dfd72683c6fd5ab766ccc5ca4dff9c9dddee1f773bc00d\": container with ID starting with a2ae77257100a13257dfd72683c6fd5ab766ccc5ca4dff9c9dddee1f773bc00d not found: ID does not exist" Aug 04 11:01:54.599632 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.599586 2535 scope.go:117] "RemoveContainer" containerID="b4095be759cff7d8c659f27c0f7ebcd74a8c400cb84871a7380b747231c86aea" Aug 04 11:01:54.607767 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.607747 2535 scope.go:117] "RemoveContainer" containerID="b4095be759cff7d8c659f27c0f7ebcd74a8c400cb84871a7380b747231c86aea" Aug 04 11:01:54.608022 ip-10-0-130-250 kubenswrapper[2535]: E0804 11:01:54.608002 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4095be759cff7d8c659f27c0f7ebcd74a8c400cb84871a7380b747231c86aea\": container with ID starting with b4095be759cff7d8c659f27c0f7ebcd74a8c400cb84871a7380b747231c86aea not found: ID does not exist" containerID="b4095be759cff7d8c659f27c0f7ebcd74a8c400cb84871a7380b747231c86aea" Aug 04 11:01:54.608092 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.608031 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4095be759cff7d8c659f27c0f7ebcd74a8c400cb84871a7380b747231c86aea"} err="failed to get container status \"b4095be759cff7d8c659f27c0f7ebcd74a8c400cb84871a7380b747231c86aea\": rpc error: code = NotFound desc = could not find container \"b4095be759cff7d8c659f27c0f7ebcd74a8c400cb84871a7380b747231c86aea\": container with ID starting with b4095be759cff7d8c659f27c0f7ebcd74a8c400cb84871a7380b747231c86aea not found: ID does not exist" Aug 04 11:01:54.608092 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.608005 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn"] Aug 04 11:01:54.613108 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.613088 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fe15eb98-6b74555b8d-cdrsn"] Aug 04 11:01:54.622164 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.622138 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv"] Aug 04 11:01:54.625960 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:54.625930 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fe15eb98-84b48f5d58-xspgv"] Aug 04 11:01:56.480284 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:56.480252 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fd57c23-d075-4661-a033-dcbbd537c291" path="/var/lib/kubelet/pods/6fd57c23-d075-4661-a033-dcbbd537c291/volumes" Aug 04 11:01:56.480648 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:56.480613 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a431d341-30e7-4cf1-a87f-93c0536c1343" path="/var/lib/kubelet/pods/a431d341-30e7-4cf1-a87f-93c0536c1343/volumes" Aug 04 11:01:56.761723 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:56.761637 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:01:56.863304 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:01:56.863267 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:02:00.807463 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:02:00.807421 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:02:00.885184 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:02:00.885151 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:02:01.087933 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:02:01.087848 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:02:01.328207 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:02:01.328178 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:02:01.436703 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:02:01.436618 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:02:01.536200 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:02:01.536165 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:02:01.672091 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:02:01.672058 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:02:07.872951 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:02:07.872869 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:02:07.975383 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:02:07.975349 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:02:08.082766 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:02:08.082733 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:02:08.236919 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:02:08.236830 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:02:14.374783 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:02:14.374747 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:02:14.442425 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:02:14.442392 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:04:01.671248 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:01.671214 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:04:17.608951 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:17.608913 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:04:17.913122 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:17.913044 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:04:18.025969 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:18.025939 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:04:21.042650 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:21.042614 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:04:21.996721 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:21.996687 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:04:22.194266 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:22.194231 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:04:22.542734 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:22.542682 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:04:22.873941 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:22.873848 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:04:22.961199 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:22.961167 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48"] Aug 04 11:04:22.961562 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:22.961548 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fd57c23-d075-4661-a033-dcbbd537c291" containerName="payload-pre-processing" Aug 04 11:04:22.961617 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:22.961565 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fd57c23-d075-4661-a033-dcbbd537c291" containerName="payload-pre-processing" Aug 04 11:04:22.961617 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:22.961586 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a431d341-30e7-4cf1-a87f-93c0536c1343" containerName="payload-processing" Aug 04 11:04:22.961617 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:22.961595 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="a431d341-30e7-4cf1-a87f-93c0536c1343" containerName="payload-processing" Aug 04 11:04:22.961717 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:22.961659 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="a431d341-30e7-4cf1-a87f-93c0536c1343" containerName="payload-processing" Aug 04 11:04:22.961717 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:22.961671 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fd57c23-d075-4661-a033-dcbbd537c291" containerName="payload-pre-processing" Aug 04 11:04:22.964674 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:22.964651 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48" Aug 04 11:04:22.966802 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:22.966781 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-a18af8fb-dockercfg-ssvp6\"" Aug 04 11:04:22.966937 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:22.966851 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-a18af8fb\"" Aug 04 11:04:22.970124 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:22.970101 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:04:22.975797 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:22.975775 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48"] Aug 04 11:04:22.977219 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:22.977197 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm"] Aug 04 11:04:22.980901 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:22.980884 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm" Aug 04 11:04:22.991192 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:22.991165 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm"] Aug 04 11:04:23.029319 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:23.029293 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3f4481b-0312-43b0-8ae9-2f00ea2ea641-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48\" (UID: \"e3f4481b-0312-43b0-8ae9-2f00ea2ea641\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48" Aug 04 11:04:23.029482 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:23.029345 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2bhb\" (UniqueName: \"kubernetes.io/projected/b1a30f66-5e8c-4aa2-b89a-d72552afcdbc-kube-api-access-w2bhb\") pod \"payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm\" (UID: \"b1a30f66-5e8c-4aa2-b89a-d72552afcdbc\") " pod="openshift-ingress/payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm" Aug 04 11:04:23.029482 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:23.029450 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr2kj\" (UniqueName: \"kubernetes.io/projected/e3f4481b-0312-43b0-8ae9-2f00ea2ea641-kube-api-access-fr2kj\") pod \"payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48\" (UID: \"e3f4481b-0312-43b0-8ae9-2f00ea2ea641\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48" Aug 04 11:04:23.029482 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:23.029478 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1a30f66-5e8c-4aa2-b89a-d72552afcdbc-plugins-config-volume\") pod \"payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm\" (UID: \"b1a30f66-5e8c-4aa2-b89a-d72552afcdbc\") " pod="openshift-ingress/payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm" Aug 04 11:04:23.131006 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:23.130921 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3f4481b-0312-43b0-8ae9-2f00ea2ea641-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48\" (UID: \"e3f4481b-0312-43b0-8ae9-2f00ea2ea641\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48" Aug 04 11:04:23.131006 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:23.130973 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2bhb\" (UniqueName: \"kubernetes.io/projected/b1a30f66-5e8c-4aa2-b89a-d72552afcdbc-kube-api-access-w2bhb\") pod \"payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm\" (UID: \"b1a30f66-5e8c-4aa2-b89a-d72552afcdbc\") " pod="openshift-ingress/payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm" Aug 04 11:04:23.131189 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:23.131097 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr2kj\" (UniqueName: \"kubernetes.io/projected/e3f4481b-0312-43b0-8ae9-2f00ea2ea641-kube-api-access-fr2kj\") pod \"payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48\" (UID: \"e3f4481b-0312-43b0-8ae9-2f00ea2ea641\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48" Aug 04 11:04:23.131189 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:23.131129 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1a30f66-5e8c-4aa2-b89a-d72552afcdbc-plugins-config-volume\") pod \"payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm\" (UID: \"b1a30f66-5e8c-4aa2-b89a-d72552afcdbc\") " pod="openshift-ingress/payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm" Aug 04 11:04:23.131836 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:23.131816 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1a30f66-5e8c-4aa2-b89a-d72552afcdbc-plugins-config-volume\") pod \"payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm\" (UID: \"b1a30f66-5e8c-4aa2-b89a-d72552afcdbc\") " pod="openshift-ingress/payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm" Aug 04 11:04:23.131933 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:23.131910 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3f4481b-0312-43b0-8ae9-2f00ea2ea641-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48\" (UID: \"e3f4481b-0312-43b0-8ae9-2f00ea2ea641\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48" Aug 04 11:04:23.144546 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:23.144514 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2bhb\" (UniqueName: \"kubernetes.io/projected/b1a30f66-5e8c-4aa2-b89a-d72552afcdbc-kube-api-access-w2bhb\") pod \"payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm\" (UID: \"b1a30f66-5e8c-4aa2-b89a-d72552afcdbc\") " pod="openshift-ingress/payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm" Aug 04 11:04:23.144675 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:23.144615 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr2kj\" (UniqueName: \"kubernetes.io/projected/e3f4481b-0312-43b0-8ae9-2f00ea2ea641-kube-api-access-fr2kj\") pod \"payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48\" (UID: \"e3f4481b-0312-43b0-8ae9-2f00ea2ea641\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48" Aug 04 11:04:23.277625 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:23.277591 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48" Aug 04 11:04:23.291902 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:23.291879 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm" Aug 04 11:04:23.424686 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:23.424655 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48"] Aug 04 11:04:23.427067 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:04:23.427037 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3f4481b_0312_43b0_8ae9_2f00ea2ea641.slice/crio-fb2ec565ad9647e71e009db880b5176d743563f1b3242a4a8afacf91b3804cae WatchSource:0}: Error finding container fb2ec565ad9647e71e009db880b5176d743563f1b3242a4a8afacf91b3804cae: Status 404 returned error can't find the container with id fb2ec565ad9647e71e009db880b5176d743563f1b3242a4a8afacf91b3804cae Aug 04 11:04:23.454885 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:23.454859 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm"] Aug 04 11:04:24.273006 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:24.272955 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm" event={"ID":"b1a30f66-5e8c-4aa2-b89a-d72552afcdbc","Type":"ContainerStarted","Data":"4c6b09f658fdfa59afd7a39176ab75ec20e44fcf6b40da88e2c597a50e0186b9"} Aug 04 11:04:24.273206 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:24.273026 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm" Aug 04 11:04:24.273206 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:24.273081 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm" event={"ID":"b1a30f66-5e8c-4aa2-b89a-d72552afcdbc","Type":"ContainerStarted","Data":"6e6444b78f9bcf2b291afa5abd92ad43de95b5e1262be1d139e182a939408320"} Aug 04 11:04:24.274705 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:24.274678 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48" event={"ID":"e3f4481b-0312-43b0-8ae9-2f00ea2ea641","Type":"ContainerStarted","Data":"6b3b7c7acb5a002830858eaeb0535d48b8f1c2e7d444dc8b2aa79833bc157e62"} Aug 04 11:04:24.274824 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:24.274726 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48" event={"ID":"e3f4481b-0312-43b0-8ae9-2f00ea2ea641","Type":"ContainerStarted","Data":"fb2ec565ad9647e71e009db880b5176d743563f1b3242a4a8afacf91b3804cae"} Aug 04 11:04:24.274824 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:24.274756 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48" Aug 04 11:04:24.289614 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:24.289562 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm" podStartSLOduration=2.289541718 podStartE2EDuration="2.289541718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:04:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:04:24.288702427 +0000 UTC m=+2202.262784981" watchObservedRunningTime="2026-08-04 11:04:24.289541718 +0000 UTC m=+2202.263624245" Aug 04 11:04:24.304873 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:24.304143 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48" podStartSLOduration=2.304125498 podStartE2EDuration="2.304125498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:04:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:04:24.303643786 +0000 UTC m=+2202.277726315" watchObservedRunningTime="2026-08-04 11:04:24.304125498 +0000 UTC m=+2202.278208026" Aug 04 11:04:35.282089 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:35.282058 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm" Aug 04 11:04:35.282571 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:35.282136 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48" Aug 04 11:04:46.108223 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:46.108175 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:04:55.730879 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:55.730842 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:04:58.329766 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:58.329734 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:04:58.431512 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:04:58.431467 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:05:03.277066 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:03.277027 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:05:40.740757 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:40.740726 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:05:40.773639 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:40.773605 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm"] Aug 04 11:05:40.773859 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:40.773835 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm" podUID="b1a30f66-5e8c-4aa2-b89a-d72552afcdbc" containerName="payload-processing" containerID="cri-o://4c6b09f658fdfa59afd7a39176ab75ec20e44fcf6b40da88e2c597a50e0186b9" gracePeriod=30 Aug 04 11:05:40.780718 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:40.780687 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48"] Aug 04 11:05:40.781120 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:40.781055 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48" podUID="e3f4481b-0312-43b0-8ae9-2f00ea2ea641" containerName="payload-pre-processing" containerID="cri-o://6b3b7c7acb5a002830858eaeb0535d48b8f1c2e7d444dc8b2aa79833bc157e62" gracePeriod=30 Aug 04 11:05:40.816437 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:40.816406 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:05:41.051365 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.051344 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm" Aug 04 11:05:41.077299 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.077274 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48" Aug 04 11:05:41.207577 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.207538 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3f4481b-0312-43b0-8ae9-2f00ea2ea641-plugins-config-volume\") pod \"e3f4481b-0312-43b0-8ae9-2f00ea2ea641\" (UID: \"e3f4481b-0312-43b0-8ae9-2f00ea2ea641\") " Aug 04 11:05:41.207779 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.207596 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1a30f66-5e8c-4aa2-b89a-d72552afcdbc-plugins-config-volume\") pod \"b1a30f66-5e8c-4aa2-b89a-d72552afcdbc\" (UID: \"b1a30f66-5e8c-4aa2-b89a-d72552afcdbc\") " Aug 04 11:05:41.207779 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.207634 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2bhb\" (UniqueName: \"kubernetes.io/projected/b1a30f66-5e8c-4aa2-b89a-d72552afcdbc-kube-api-access-w2bhb\") pod \"b1a30f66-5e8c-4aa2-b89a-d72552afcdbc\" (UID: \"b1a30f66-5e8c-4aa2-b89a-d72552afcdbc\") " Aug 04 11:05:41.207902 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.207858 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fr2kj\" (UniqueName: \"kubernetes.io/projected/e3f4481b-0312-43b0-8ae9-2f00ea2ea641-kube-api-access-fr2kj\") pod \"e3f4481b-0312-43b0-8ae9-2f00ea2ea641\" (UID: \"e3f4481b-0312-43b0-8ae9-2f00ea2ea641\") " Aug 04 11:05:41.207958 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.207920 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3f4481b-0312-43b0-8ae9-2f00ea2ea641-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e3f4481b-0312-43b0-8ae9-2f00ea2ea641" (UID: "e3f4481b-0312-43b0-8ae9-2f00ea2ea641"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:05:41.208013 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.207952 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1a30f66-5e8c-4aa2-b89a-d72552afcdbc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b1a30f66-5e8c-4aa2-b89a-d72552afcdbc" (UID: "b1a30f66-5e8c-4aa2-b89a-d72552afcdbc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:05:41.208204 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.208182 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3f4481b-0312-43b0-8ae9-2f00ea2ea641-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:05:41.208267 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.208204 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1a30f66-5e8c-4aa2-b89a-d72552afcdbc-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:05:41.209743 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.209719 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1a30f66-5e8c-4aa2-b89a-d72552afcdbc-kube-api-access-w2bhb" (OuterVolumeSpecName: "kube-api-access-w2bhb") pod "b1a30f66-5e8c-4aa2-b89a-d72552afcdbc" (UID: "b1a30f66-5e8c-4aa2-b89a-d72552afcdbc"). InnerVolumeSpecName "kube-api-access-w2bhb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 11:05:41.209833 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.209805 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3f4481b-0312-43b0-8ae9-2f00ea2ea641-kube-api-access-fr2kj" (OuterVolumeSpecName: "kube-api-access-fr2kj") pod "e3f4481b-0312-43b0-8ae9-2f00ea2ea641" (UID: "e3f4481b-0312-43b0-8ae9-2f00ea2ea641"). InnerVolumeSpecName "kube-api-access-fr2kj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 11:05:41.309579 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.309467 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fr2kj\" (UniqueName: \"kubernetes.io/projected/e3f4481b-0312-43b0-8ae9-2f00ea2ea641-kube-api-access-fr2kj\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:05:41.309579 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.309522 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2bhb\" (UniqueName: \"kubernetes.io/projected/b1a30f66-5e8c-4aa2-b89a-d72552afcdbc-kube-api-access-w2bhb\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:05:41.630318 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.630232 2535 generic.go:356] "Generic (PLEG): container finished" podID="b1a30f66-5e8c-4aa2-b89a-d72552afcdbc" containerID="4c6b09f658fdfa59afd7a39176ab75ec20e44fcf6b40da88e2c597a50e0186b9" exitCode=0 Aug 04 11:05:41.630318 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.630275 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm" Aug 04 11:05:41.630318 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.630286 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm" event={"ID":"b1a30f66-5e8c-4aa2-b89a-d72552afcdbc","Type":"ContainerDied","Data":"4c6b09f658fdfa59afd7a39176ab75ec20e44fcf6b40da88e2c597a50e0186b9"} Aug 04 11:05:41.630614 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.630345 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm" event={"ID":"b1a30f66-5e8c-4aa2-b89a-d72552afcdbc","Type":"ContainerDied","Data":"6e6444b78f9bcf2b291afa5abd92ad43de95b5e1262be1d139e182a939408320"} Aug 04 11:05:41.630614 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.630377 2535 scope.go:117] "RemoveContainer" containerID="4c6b09f658fdfa59afd7a39176ab75ec20e44fcf6b40da88e2c597a50e0186b9" Aug 04 11:05:41.631674 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.631651 2535 generic.go:356] "Generic (PLEG): container finished" podID="e3f4481b-0312-43b0-8ae9-2f00ea2ea641" containerID="6b3b7c7acb5a002830858eaeb0535d48b8f1c2e7d444dc8b2aa79833bc157e62" exitCode=0 Aug 04 11:05:41.631800 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.631692 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48" Aug 04 11:05:41.631800 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.631705 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48" event={"ID":"e3f4481b-0312-43b0-8ae9-2f00ea2ea641","Type":"ContainerDied","Data":"6b3b7c7acb5a002830858eaeb0535d48b8f1c2e7d444dc8b2aa79833bc157e62"} Aug 04 11:05:41.631800 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.631738 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48" event={"ID":"e3f4481b-0312-43b0-8ae9-2f00ea2ea641","Type":"ContainerDied","Data":"fb2ec565ad9647e71e009db880b5176d743563f1b3242a4a8afacf91b3804cae"} Aug 04 11:05:41.640703 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.640679 2535 scope.go:117] "RemoveContainer" containerID="4c6b09f658fdfa59afd7a39176ab75ec20e44fcf6b40da88e2c597a50e0186b9" Aug 04 11:05:41.640961 ip-10-0-130-250 kubenswrapper[2535]: E0804 11:05:41.640938 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c6b09f658fdfa59afd7a39176ab75ec20e44fcf6b40da88e2c597a50e0186b9\": container with ID starting with 4c6b09f658fdfa59afd7a39176ab75ec20e44fcf6b40da88e2c597a50e0186b9 not found: ID does not exist" containerID="4c6b09f658fdfa59afd7a39176ab75ec20e44fcf6b40da88e2c597a50e0186b9" Aug 04 11:05:41.641015 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.640969 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c6b09f658fdfa59afd7a39176ab75ec20e44fcf6b40da88e2c597a50e0186b9"} err="failed to get container status \"4c6b09f658fdfa59afd7a39176ab75ec20e44fcf6b40da88e2c597a50e0186b9\": rpc error: code = NotFound desc = could not find container \"4c6b09f658fdfa59afd7a39176ab75ec20e44fcf6b40da88e2c597a50e0186b9\": container with ID starting with 4c6b09f658fdfa59afd7a39176ab75ec20e44fcf6b40da88e2c597a50e0186b9 not found: ID does not exist" Aug 04 11:05:41.641015 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.640985 2535 scope.go:117] "RemoveContainer" containerID="6b3b7c7acb5a002830858eaeb0535d48b8f1c2e7d444dc8b2aa79833bc157e62" Aug 04 11:05:41.648725 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.648707 2535 scope.go:117] "RemoveContainer" containerID="6b3b7c7acb5a002830858eaeb0535d48b8f1c2e7d444dc8b2aa79833bc157e62" Aug 04 11:05:41.648961 ip-10-0-130-250 kubenswrapper[2535]: E0804 11:05:41.648943 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b3b7c7acb5a002830858eaeb0535d48b8f1c2e7d444dc8b2aa79833bc157e62\": container with ID starting with 6b3b7c7acb5a002830858eaeb0535d48b8f1c2e7d444dc8b2aa79833bc157e62 not found: ID does not exist" containerID="6b3b7c7acb5a002830858eaeb0535d48b8f1c2e7d444dc8b2aa79833bc157e62" Aug 04 11:05:41.649006 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.648972 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b3b7c7acb5a002830858eaeb0535d48b8f1c2e7d444dc8b2aa79833bc157e62"} err="failed to get container status \"6b3b7c7acb5a002830858eaeb0535d48b8f1c2e7d444dc8b2aa79833bc157e62\": rpc error: code = NotFound desc = could not find container \"6b3b7c7acb5a002830858eaeb0535d48b8f1c2e7d444dc8b2aa79833bc157e62\": container with ID starting with 6b3b7c7acb5a002830858eaeb0535d48b8f1c2e7d444dc8b2aa79833bc157e62 not found: ID does not exist" Aug 04 11:05:41.651456 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.651436 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm"] Aug 04 11:05:41.655768 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.655748 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a18af8fb-74c7574586-hxxhm"] Aug 04 11:05:41.668931 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.668908 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48"] Aug 04 11:05:41.674014 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.673993 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a18af8fb-8484b6b86d-kfw48"] Aug 04 11:05:41.841780 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.841744 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:05:41.942015 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:41.941940 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:05:42.480977 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:42.480945 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1a30f66-5e8c-4aa2-b89a-d72552afcdbc" path="/var/lib/kubelet/pods/b1a30f66-5e8c-4aa2-b89a-d72552afcdbc/volumes" Aug 04 11:05:42.481296 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:42.481283 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3f4481b-0312-43b0-8ae9-2f00ea2ea641" path="/var/lib/kubelet/pods/e3f4481b-0312-43b0-8ae9-2f00ea2ea641/volumes" Aug 04 11:05:44.209286 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:44.209249 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:05:49.272198 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:05:49.272162 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:06:26.115767 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:06:26.115731 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:06:27.999653 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:06:27.999595 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:06:32.297159 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:06:32.297106 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:06:32.366948 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:06:32.366907 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:06:32.629675 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:06:32.629590 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:06:32.898583 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:06:32.898502 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:06:44.259434 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:06:44.259381 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:06:44.354610 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:06:44.354574 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:06:44.455897 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:06:44.455865 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:06:45.075811 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:06:45.075766 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:06:45.208658 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:06:45.208618 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:06:45.313221 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:06:45.313184 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:06:46.583724 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:06:46.583688 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:06:53.345574 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:06:53.345543 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:06:53.440932 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:06:53.440892 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:06:54.045093 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:06:54.045054 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:07:00.310212 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:00.310175 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:07:00.408995 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:00.408964 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:07:00.535548 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:00.535514 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:07:11.076359 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:11.076310 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:07:11.290418 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:11.290377 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:07:20.862456 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:20.862415 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:07:20.942702 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:20.942664 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:07:32.263221 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:32.263191 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:07:38.544528 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:38.544472 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:07:38.643776 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:38.643744 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:07:38.900672 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:38.900584 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:07:39.228706 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:39.228627 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:07:39.369723 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:39.369689 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:07:42.071899 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:42.071850 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:07:42.169071 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:42.169040 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:07:43.197169 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:43.197128 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:07:43.686388 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:43.686355 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:07:43.792472 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:43.792435 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:07:43.814455 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:43.814427 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz"] Aug 04 11:07:43.814800 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:43.814787 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3f4481b-0312-43b0-8ae9-2f00ea2ea641" containerName="payload-pre-processing" Aug 04 11:07:43.814843 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:43.814801 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3f4481b-0312-43b0-8ae9-2f00ea2ea641" containerName="payload-pre-processing" Aug 04 11:07:43.814843 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:43.814809 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1a30f66-5e8c-4aa2-b89a-d72552afcdbc" containerName="payload-processing" Aug 04 11:07:43.814843 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:43.814815 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1a30f66-5e8c-4aa2-b89a-d72552afcdbc" containerName="payload-processing" Aug 04 11:07:43.814939 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:43.814869 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3f4481b-0312-43b0-8ae9-2f00ea2ea641" containerName="payload-pre-processing" Aug 04 11:07:43.814939 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:43.814878 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1a30f66-5e8c-4aa2-b89a-d72552afcdbc" containerName="payload-processing" Aug 04 11:07:43.817755 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:43.817741 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz" Aug 04 11:07:43.819710 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:43.819688 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-78f47f\"" Aug 04 11:07:43.827752 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:43.827730 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz"] Aug 04 11:07:43.856277 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:43.856250 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq"] Aug 04 11:07:43.856609 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:43.856585 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/060dfc25-87aa-4d07-98f7-ea62af1baea5-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz\" (UID: \"060dfc25-87aa-4d07-98f7-ea62af1baea5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz" Aug 04 11:07:43.856697 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:43.856641 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frg5t\" (UniqueName: \"kubernetes.io/projected/060dfc25-87aa-4d07-98f7-ea62af1baea5-kube-api-access-frg5t\") pod \"payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz\" (UID: \"060dfc25-87aa-4d07-98f7-ea62af1baea5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz" Aug 04 11:07:43.859702 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:43.859685 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq" Aug 04 11:07:43.861730 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:43.861709 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-78f47f-dockercfg-sc5nr\"" Aug 04 11:07:43.871287 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:43.871267 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq"] Aug 04 11:07:43.957894 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:43.957819 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghkzj\" (UniqueName: \"kubernetes.io/projected/26b81f19-f2e3-4727-ad02-163ec89e9673-kube-api-access-ghkzj\") pod \"payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq\" (UID: \"26b81f19-f2e3-4727-ad02-163ec89e9673\") " pod="openshift-ingress/payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq" Aug 04 11:07:43.957894 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:43.957886 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/060dfc25-87aa-4d07-98f7-ea62af1baea5-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz\" (UID: \"060dfc25-87aa-4d07-98f7-ea62af1baea5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz" Aug 04 11:07:43.958074 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:43.957942 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26b81f19-f2e3-4727-ad02-163ec89e9673-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq\" (UID: \"26b81f19-f2e3-4727-ad02-163ec89e9673\") " pod="openshift-ingress/payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq" Aug 04 11:07:43.958074 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:43.957981 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frg5t\" (UniqueName: \"kubernetes.io/projected/060dfc25-87aa-4d07-98f7-ea62af1baea5-kube-api-access-frg5t\") pod \"payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz\" (UID: \"060dfc25-87aa-4d07-98f7-ea62af1baea5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz" Aug 04 11:07:43.958638 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:43.958620 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/060dfc25-87aa-4d07-98f7-ea62af1baea5-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz\" (UID: \"060dfc25-87aa-4d07-98f7-ea62af1baea5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz" Aug 04 11:07:43.966413 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:43.966384 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frg5t\" (UniqueName: \"kubernetes.io/projected/060dfc25-87aa-4d07-98f7-ea62af1baea5-kube-api-access-frg5t\") pod \"payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz\" (UID: \"060dfc25-87aa-4d07-98f7-ea62af1baea5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz" Aug 04 11:07:44.059143 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:44.059113 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26b81f19-f2e3-4727-ad02-163ec89e9673-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq\" (UID: \"26b81f19-f2e3-4727-ad02-163ec89e9673\") " pod="openshift-ingress/payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq" Aug 04 11:07:44.059304 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:44.059180 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghkzj\" (UniqueName: \"kubernetes.io/projected/26b81f19-f2e3-4727-ad02-163ec89e9673-kube-api-access-ghkzj\") pod \"payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq\" (UID: \"26b81f19-f2e3-4727-ad02-163ec89e9673\") " pod="openshift-ingress/payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq" Aug 04 11:07:44.059880 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:44.059862 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26b81f19-f2e3-4727-ad02-163ec89e9673-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq\" (UID: \"26b81f19-f2e3-4727-ad02-163ec89e9673\") " pod="openshift-ingress/payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq" Aug 04 11:07:44.067124 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:44.067106 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghkzj\" (UniqueName: \"kubernetes.io/projected/26b81f19-f2e3-4727-ad02-163ec89e9673-kube-api-access-ghkzj\") pod \"payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq\" (UID: \"26b81f19-f2e3-4727-ad02-163ec89e9673\") " pod="openshift-ingress/payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq" Aug 04 11:07:44.129576 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:44.129544 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz" Aug 04 11:07:44.171505 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:44.171462 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq" Aug 04 11:07:44.293969 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:44.293935 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz"] Aug 04 11:07:44.295084 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:07:44.295051 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod060dfc25_87aa_4d07_98f7_ea62af1baea5.slice/crio-7e2f8057b3d460cec599e7738d7a48c6b631948b47599a39d195456c67cd8ad3 WatchSource:0}: Error finding container 7e2f8057b3d460cec599e7738d7a48c6b631948b47599a39d195456c67cd8ad3: Status 404 returned error can't find the container with id 7e2f8057b3d460cec599e7738d7a48c6b631948b47599a39d195456c67cd8ad3 Aug 04 11:07:44.327903 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:44.327866 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq"] Aug 04 11:07:44.330695 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:07:44.330663 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26b81f19_f2e3_4727_ad02_163ec89e9673.slice/crio-4b38ae86fd50ff44c2871c0bf7a6d158d59809cfc50c53233bc2cec09398b5a3 WatchSource:0}: Error finding container 4b38ae86fd50ff44c2871c0bf7a6d158d59809cfc50c53233bc2cec09398b5a3: Status 404 returned error can't find the container with id 4b38ae86fd50ff44c2871c0bf7a6d158d59809cfc50c53233bc2cec09398b5a3 Aug 04 11:07:45.200766 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:45.200719 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq" event={"ID":"26b81f19-f2e3-4727-ad02-163ec89e9673","Type":"ContainerStarted","Data":"456a1e46f80c59297fc16479238ad713afa98a5efcf576c300138703756a4fa3"} Aug 04 11:07:45.200959 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:45.200828 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq" Aug 04 11:07:45.200959 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:45.200856 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq" event={"ID":"26b81f19-f2e3-4727-ad02-163ec89e9673","Type":"ContainerStarted","Data":"4b38ae86fd50ff44c2871c0bf7a6d158d59809cfc50c53233bc2cec09398b5a3"} Aug 04 11:07:45.202268 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:45.202245 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz" event={"ID":"060dfc25-87aa-4d07-98f7-ea62af1baea5","Type":"ContainerStarted","Data":"2a7d18b6fad39d4b426c5f4298d946647e54bdad3a6728cb66fdac2463304ec8"} Aug 04 11:07:45.202404 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:45.202279 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz" event={"ID":"060dfc25-87aa-4d07-98f7-ea62af1baea5","Type":"ContainerStarted","Data":"7e2f8057b3d460cec599e7738d7a48c6b631948b47599a39d195456c67cd8ad3"} Aug 04 11:07:45.202404 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:45.202379 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz" Aug 04 11:07:45.219374 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:45.219278 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq" podStartSLOduration=2.219261126 podStartE2EDuration="2.219261126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:07:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:07:45.218514367 +0000 UTC m=+2403.192596915" watchObservedRunningTime="2026-08-04 11:07:45.219261126 +0000 UTC m=+2403.193343658" Aug 04 11:07:45.236161 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:45.236116 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz" podStartSLOduration=2.236102839 podStartE2EDuration="2.236102839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:07:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:07:45.23385374 +0000 UTC m=+2403.207936257" watchObservedRunningTime="2026-08-04 11:07:45.236102839 +0000 UTC m=+2403.210185366" Aug 04 11:07:56.208961 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:56.208920 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz" Aug 04 11:07:56.209431 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:07:56.209000 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq" Aug 04 11:08:07.016110 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:07.016026 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:08:07.335268 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:07.335230 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:08:07.656804 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:07.656734 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:08:07.823867 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:07.823830 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:08:11.111328 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:11.111268 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:08:11.206971 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:11.206941 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:08:12.237061 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:12.237009 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:08:12.428248 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:12.428214 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:08:12.741065 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:12.741030 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:08:12.842897 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:12.842866 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:08:12.846766 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:12.846739 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn"] Aug 04 11:08:12.850347 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:12.850328 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn" Aug 04 11:08:12.852445 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:12.852429 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-1c7c26-dockercfg-fvrsx\"" Aug 04 11:08:12.852445 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:12.852439 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-1c7c26\"" Aug 04 11:08:12.861165 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:12.861143 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn"] Aug 04 11:08:12.865827 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:12.865804 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c"] Aug 04 11:08:12.869202 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:12.869186 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c" Aug 04 11:08:12.880750 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:12.880730 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c"] Aug 04 11:08:12.924245 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:12.924211 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxs27\" (UniqueName: \"kubernetes.io/projected/f6615509-6990-4059-b3f3-4aad3cd7a366-kube-api-access-jxs27\") pod \"payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn\" (UID: \"f6615509-6990-4059-b3f3-4aad3cd7a366\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn" Aug 04 11:08:12.924385 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:12.924263 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6615509-6990-4059-b3f3-4aad3cd7a366-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn\" (UID: \"f6615509-6990-4059-b3f3-4aad3cd7a366\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn" Aug 04 11:08:13.025694 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:13.025667 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6615509-6990-4059-b3f3-4aad3cd7a366-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn\" (UID: \"f6615509-6990-4059-b3f3-4aad3cd7a366\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn" Aug 04 11:08:13.025852 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:13.025709 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztj2b\" (UniqueName: \"kubernetes.io/projected/3e4b61c8-b17b-41dd-93df-e1e22f13a8cc-kube-api-access-ztj2b\") pod \"payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c\" (UID: \"3e4b61c8-b17b-41dd-93df-e1e22f13a8cc\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c" Aug 04 11:08:13.025852 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:13.025765 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e4b61c8-b17b-41dd-93df-e1e22f13a8cc-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c\" (UID: \"3e4b61c8-b17b-41dd-93df-e1e22f13a8cc\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c" Aug 04 11:08:13.025852 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:13.025795 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxs27\" (UniqueName: \"kubernetes.io/projected/f6615509-6990-4059-b3f3-4aad3cd7a366-kube-api-access-jxs27\") pod \"payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn\" (UID: \"f6615509-6990-4059-b3f3-4aad3cd7a366\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn" Aug 04 11:08:13.026396 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:13.026377 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6615509-6990-4059-b3f3-4aad3cd7a366-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn\" (UID: \"f6615509-6990-4059-b3f3-4aad3cd7a366\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn" Aug 04 11:08:13.033735 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:13.033708 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxs27\" (UniqueName: \"kubernetes.io/projected/f6615509-6990-4059-b3f3-4aad3cd7a366-kube-api-access-jxs27\") pod \"payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn\" (UID: \"f6615509-6990-4059-b3f3-4aad3cd7a366\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn" Aug 04 11:08:13.127094 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:13.127063 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztj2b\" (UniqueName: \"kubernetes.io/projected/3e4b61c8-b17b-41dd-93df-e1e22f13a8cc-kube-api-access-ztj2b\") pod \"payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c\" (UID: \"3e4b61c8-b17b-41dd-93df-e1e22f13a8cc\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c" Aug 04 11:08:13.127256 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:13.127124 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e4b61c8-b17b-41dd-93df-e1e22f13a8cc-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c\" (UID: \"3e4b61c8-b17b-41dd-93df-e1e22f13a8cc\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c" Aug 04 11:08:13.127777 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:13.127758 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e4b61c8-b17b-41dd-93df-e1e22f13a8cc-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c\" (UID: \"3e4b61c8-b17b-41dd-93df-e1e22f13a8cc\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c" Aug 04 11:08:13.134335 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:13.134316 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztj2b\" (UniqueName: \"kubernetes.io/projected/3e4b61c8-b17b-41dd-93df-e1e22f13a8cc-kube-api-access-ztj2b\") pod \"payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c\" (UID: \"3e4b61c8-b17b-41dd-93df-e1e22f13a8cc\") " pod="openshift-ingress/payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c" Aug 04 11:08:13.162259 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:13.162239 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn" Aug 04 11:08:13.181507 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:13.181466 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c" Aug 04 11:08:13.514675 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:13.514642 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn"] Aug 04 11:08:13.518833 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:08:13.518804 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6615509_6990_4059_b3f3_4aad3cd7a366.slice/crio-97eca87bf24d82a35bb6ecafa7c3c958f2da95fbbda7b44de7fef404e8852842 WatchSource:0}: Error finding container 97eca87bf24d82a35bb6ecafa7c3c958f2da95fbbda7b44de7fef404e8852842: Status 404 returned error can't find the container with id 97eca87bf24d82a35bb6ecafa7c3c958f2da95fbbda7b44de7fef404e8852842 Aug 04 11:08:13.544586 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:13.544565 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c"] Aug 04 11:08:13.550606 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:08:13.550575 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e4b61c8_b17b_41dd_93df_e1e22f13a8cc.slice/crio-b4f6319ed5630464a5b053ac53ec554f877cf87ee9381346964b27d3015e6aeb WatchSource:0}: Error finding container b4f6319ed5630464a5b053ac53ec554f877cf87ee9381346964b27d3015e6aeb: Status 404 returned error can't find the container with id b4f6319ed5630464a5b053ac53ec554f877cf87ee9381346964b27d3015e6aeb Aug 04 11:08:14.338855 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:14.338819 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn" event={"ID":"f6615509-6990-4059-b3f3-4aad3cd7a366","Type":"ContainerStarted","Data":"829c9b82abdaeb78caa3d20b3a565ad69e4a9a8b920803c3a2933b5b753064a1"} Aug 04 11:08:14.339048 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:14.338879 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn" event={"ID":"f6615509-6990-4059-b3f3-4aad3cd7a366","Type":"ContainerStarted","Data":"97eca87bf24d82a35bb6ecafa7c3c958f2da95fbbda7b44de7fef404e8852842"} Aug 04 11:08:14.339048 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:14.339005 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn" Aug 04 11:08:14.340338 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:14.340316 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c" event={"ID":"3e4b61c8-b17b-41dd-93df-e1e22f13a8cc","Type":"ContainerStarted","Data":"f18dae808341e8e10fd83041e7300917c02166c8d45bfd6b366159f9313399a2"} Aug 04 11:08:14.340436 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:14.340356 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c" event={"ID":"3e4b61c8-b17b-41dd-93df-e1e22f13a8cc","Type":"ContainerStarted","Data":"b4f6319ed5630464a5b053ac53ec554f877cf87ee9381346964b27d3015e6aeb"} Aug 04 11:08:14.340511 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:14.340437 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c" Aug 04 11:08:14.355012 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:14.354975 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn" podStartSLOduration=2.354961609 podStartE2EDuration="2.354961609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:08:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:08:14.35347548 +0000 UTC m=+2432.327557998" watchObservedRunningTime="2026-08-04 11:08:14.354961609 +0000 UTC m=+2432.329044149" Aug 04 11:08:14.370570 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:14.370531 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c" podStartSLOduration=2.370519974 podStartE2EDuration="2.370519974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:08:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:08:14.369608517 +0000 UTC m=+2432.343691079" watchObservedRunningTime="2026-08-04 11:08:14.370519974 +0000 UTC m=+2432.344602495" Aug 04 11:08:25.347286 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:25.347230 2535 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn" podUID="f6615509-6990-4059-b3f3-4aad3cd7a366" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.51:9004: connect: connection refused" Aug 04 11:08:25.347765 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:25.347613 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c" Aug 04 11:08:35.346427 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:35.346394 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn" Aug 04 11:08:36.129056 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:36.129014 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:08:47.569117 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:08:47.569085 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:09:16.346961 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:09:16.346920 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:09:32.446035 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:09:32.445998 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:09:38.757460 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:09:38.757388 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:09:38.830528 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:09:38.830469 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:10:01.748670 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:01.748633 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:10:01.970596 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:01.970564 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:10:06.559583 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:06.559546 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:10:06.588239 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:06.588205 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq"] Aug 04 11:10:06.588478 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:06.588446 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq" podUID="26b81f19-f2e3-4727-ad02-163ec89e9673" containerName="payload-processing" containerID="cri-o://456a1e46f80c59297fc16479238ad713afa98a5efcf576c300138703756a4fa3" gracePeriod=30 Aug 04 11:10:06.594121 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:06.594091 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz"] Aug 04 11:10:06.594586 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:06.594460 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz" podUID="060dfc25-87aa-4d07-98f7-ea62af1baea5" containerName="payload-pre-processing" containerID="cri-o://2a7d18b6fad39d4b426c5f4298d946647e54bdad3a6728cb66fdac2463304ec8" gracePeriod=30 Aug 04 11:10:06.659901 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:06.659871 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:10:06.727113 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:06.727080 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:10:06.855803 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:06.855777 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq" Aug 04 11:10:06.870315 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:06.870286 2535 generic.go:356] "Generic (PLEG): container finished" podID="060dfc25-87aa-4d07-98f7-ea62af1baea5" containerID="2a7d18b6fad39d4b426c5f4298d946647e54bdad3a6728cb66fdac2463304ec8" exitCode=0 Aug 04 11:10:06.870412 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:06.870363 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz" event={"ID":"060dfc25-87aa-4d07-98f7-ea62af1baea5","Type":"ContainerDied","Data":"2a7d18b6fad39d4b426c5f4298d946647e54bdad3a6728cb66fdac2463304ec8"} Aug 04 11:10:06.871779 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:06.871757 2535 generic.go:356] "Generic (PLEG): container finished" podID="26b81f19-f2e3-4727-ad02-163ec89e9673" containerID="456a1e46f80c59297fc16479238ad713afa98a5efcf576c300138703756a4fa3" exitCode=0 Aug 04 11:10:06.871874 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:06.871786 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq" event={"ID":"26b81f19-f2e3-4727-ad02-163ec89e9673","Type":"ContainerDied","Data":"456a1e46f80c59297fc16479238ad713afa98a5efcf576c300138703756a4fa3"} Aug 04 11:10:06.871874 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:06.871806 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq" Aug 04 11:10:06.871874 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:06.871834 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq" event={"ID":"26b81f19-f2e3-4727-ad02-163ec89e9673","Type":"ContainerDied","Data":"4b38ae86fd50ff44c2871c0bf7a6d158d59809cfc50c53233bc2cec09398b5a3"} Aug 04 11:10:06.871874 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:06.871864 2535 scope.go:117] "RemoveContainer" containerID="456a1e46f80c59297fc16479238ad713afa98a5efcf576c300138703756a4fa3" Aug 04 11:10:06.882462 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:06.882435 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz" Aug 04 11:10:06.883292 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:06.882997 2535 scope.go:117] "RemoveContainer" containerID="456a1e46f80c59297fc16479238ad713afa98a5efcf576c300138703756a4fa3" Aug 04 11:10:06.883696 ip-10-0-130-250 kubenswrapper[2535]: E0804 11:10:06.883654 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"456a1e46f80c59297fc16479238ad713afa98a5efcf576c300138703756a4fa3\": container with ID starting with 456a1e46f80c59297fc16479238ad713afa98a5efcf576c300138703756a4fa3 not found: ID does not exist" containerID="456a1e46f80c59297fc16479238ad713afa98a5efcf576c300138703756a4fa3" Aug 04 11:10:06.883776 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:06.883705 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"456a1e46f80c59297fc16479238ad713afa98a5efcf576c300138703756a4fa3"} err="failed to get container status \"456a1e46f80c59297fc16479238ad713afa98a5efcf576c300138703756a4fa3\": rpc error: code = NotFound desc = could not find container \"456a1e46f80c59297fc16479238ad713afa98a5efcf576c300138703756a4fa3\": container with ID starting with 456a1e46f80c59297fc16479238ad713afa98a5efcf576c300138703756a4fa3 not found: ID does not exist" Aug 04 11:10:06.936536 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:06.936507 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/060dfc25-87aa-4d07-98f7-ea62af1baea5-plugins-config-volume\") pod \"060dfc25-87aa-4d07-98f7-ea62af1baea5\" (UID: \"060dfc25-87aa-4d07-98f7-ea62af1baea5\") " Aug 04 11:10:06.936692 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:06.936598 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghkzj\" (UniqueName: \"kubernetes.io/projected/26b81f19-f2e3-4727-ad02-163ec89e9673-kube-api-access-ghkzj\") pod \"26b81f19-f2e3-4727-ad02-163ec89e9673\" (UID: \"26b81f19-f2e3-4727-ad02-163ec89e9673\") " Aug 04 11:10:06.936692 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:06.936644 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frg5t\" (UniqueName: \"kubernetes.io/projected/060dfc25-87aa-4d07-98f7-ea62af1baea5-kube-api-access-frg5t\") pod \"060dfc25-87aa-4d07-98f7-ea62af1baea5\" (UID: \"060dfc25-87aa-4d07-98f7-ea62af1baea5\") " Aug 04 11:10:06.936838 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:06.936707 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26b81f19-f2e3-4727-ad02-163ec89e9673-plugins-config-volume\") pod \"26b81f19-f2e3-4727-ad02-163ec89e9673\" (UID: \"26b81f19-f2e3-4727-ad02-163ec89e9673\") " Aug 04 11:10:06.936898 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:06.936825 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/060dfc25-87aa-4d07-98f7-ea62af1baea5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "060dfc25-87aa-4d07-98f7-ea62af1baea5" (UID: "060dfc25-87aa-4d07-98f7-ea62af1baea5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:10:06.937142 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:06.937116 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26b81f19-f2e3-4727-ad02-163ec89e9673-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "26b81f19-f2e3-4727-ad02-163ec89e9673" (UID: "26b81f19-f2e3-4727-ad02-163ec89e9673"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:10:06.937142 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:06.937127 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/060dfc25-87aa-4d07-98f7-ea62af1baea5-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:10:06.938625 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:06.938603 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26b81f19-f2e3-4727-ad02-163ec89e9673-kube-api-access-ghkzj" (OuterVolumeSpecName: "kube-api-access-ghkzj") pod "26b81f19-f2e3-4727-ad02-163ec89e9673" (UID: "26b81f19-f2e3-4727-ad02-163ec89e9673"). InnerVolumeSpecName "kube-api-access-ghkzj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 11:10:06.938694 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:06.938625 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/060dfc25-87aa-4d07-98f7-ea62af1baea5-kube-api-access-frg5t" (OuterVolumeSpecName: "kube-api-access-frg5t") pod "060dfc25-87aa-4d07-98f7-ea62af1baea5" (UID: "060dfc25-87aa-4d07-98f7-ea62af1baea5"). InnerVolumeSpecName "kube-api-access-frg5t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 11:10:07.037736 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:07.037709 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26b81f19-f2e3-4727-ad02-163ec89e9673-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:10:07.037736 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:07.037732 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghkzj\" (UniqueName: \"kubernetes.io/projected/26b81f19-f2e3-4727-ad02-163ec89e9673-kube-api-access-ghkzj\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:10:07.037736 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:07.037742 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frg5t\" (UniqueName: \"kubernetes.io/projected/060dfc25-87aa-4d07-98f7-ea62af1baea5-kube-api-access-frg5t\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:10:07.119959 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:07.119888 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:10:07.194196 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:07.194167 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq"] Aug 04 11:10:07.198607 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:07.198585 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-78f47f-68dc868978-2bdpq"] Aug 04 11:10:07.879720 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:07.879633 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz" Aug 04 11:10:07.879720 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:07.879682 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz" event={"ID":"060dfc25-87aa-4d07-98f7-ea62af1baea5","Type":"ContainerDied","Data":"7e2f8057b3d460cec599e7738d7a48c6b631948b47599a39d195456c67cd8ad3"} Aug 04 11:10:07.880262 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:07.879747 2535 scope.go:117] "RemoveContainer" containerID="2a7d18b6fad39d4b426c5f4298d946647e54bdad3a6728cb66fdac2463304ec8" Aug 04 11:10:07.902910 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:07.902885 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz"] Aug 04 11:10:07.906708 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:07.906687 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-78f47f-5b95cb944-zd8tz"] Aug 04 11:10:08.480966 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:08.480929 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="060dfc25-87aa-4d07-98f7-ea62af1baea5" path="/var/lib/kubelet/pods/060dfc25-87aa-4d07-98f7-ea62af1baea5/volumes" Aug 04 11:10:08.481289 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:08.481277 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26b81f19-f2e3-4727-ad02-163ec89e9673" path="/var/lib/kubelet/pods/26b81f19-f2e3-4727-ad02-163ec89e9673/volumes" Aug 04 11:10:12.187838 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:12.187800 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:10:18.479655 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:18.479613 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:10:18.757375 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:18.757294 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:10:18.973840 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:18.973806 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:10:29.595123 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:29.595078 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:10:29.598859 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:29.598829 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c"] Aug 04 11:10:29.599290 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:29.599238 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c" podUID="3e4b61c8-b17b-41dd-93df-e1e22f13a8cc" containerName="payload-processing" containerID="cri-o://f18dae808341e8e10fd83041e7300917c02166c8d45bfd6b366159f9313399a2" gracePeriod=30 Aug 04 11:10:29.604525 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:29.604472 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn"] Aug 04 11:10:29.604743 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:29.604720 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn" podUID="f6615509-6990-4059-b3f3-4aad3cd7a366" containerName="payload-pre-processing" containerID="cri-o://829c9b82abdaeb78caa3d20b3a565ad69e4a9a8b920803c3a2933b5b753064a1" gracePeriod=30 Aug 04 11:10:29.695420 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:29.695390 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:10:29.796670 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:29.796631 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:10:29.856693 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:29.856630 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c" Aug 04 11:10:29.884236 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:29.884213 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn" Aug 04 11:10:29.993009 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:29.992972 2535 generic.go:356] "Generic (PLEG): container finished" podID="3e4b61c8-b17b-41dd-93df-e1e22f13a8cc" containerID="f18dae808341e8e10fd83041e7300917c02166c8d45bfd6b366159f9313399a2" exitCode=0 Aug 04 11:10:29.993192 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:29.993026 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c" Aug 04 11:10:29.993192 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:29.993051 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c" event={"ID":"3e4b61c8-b17b-41dd-93df-e1e22f13a8cc","Type":"ContainerDied","Data":"f18dae808341e8e10fd83041e7300917c02166c8d45bfd6b366159f9313399a2"} Aug 04 11:10:29.993192 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:29.993131 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c" event={"ID":"3e4b61c8-b17b-41dd-93df-e1e22f13a8cc","Type":"ContainerDied","Data":"b4f6319ed5630464a5b053ac53ec554f877cf87ee9381346964b27d3015e6aeb"} Aug 04 11:10:29.993192 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:29.993163 2535 scope.go:117] "RemoveContainer" containerID="f18dae808341e8e10fd83041e7300917c02166c8d45bfd6b366159f9313399a2" Aug 04 11:10:29.994238 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:29.994218 2535 generic.go:356] "Generic (PLEG): container finished" podID="f6615509-6990-4059-b3f3-4aad3cd7a366" containerID="829c9b82abdaeb78caa3d20b3a565ad69e4a9a8b920803c3a2933b5b753064a1" exitCode=0 Aug 04 11:10:29.994322 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:29.994251 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn" Aug 04 11:10:29.994322 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:29.994302 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn" event={"ID":"f6615509-6990-4059-b3f3-4aad3cd7a366","Type":"ContainerDied","Data":"829c9b82abdaeb78caa3d20b3a565ad69e4a9a8b920803c3a2933b5b753064a1"} Aug 04 11:10:29.994396 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:29.994338 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn" event={"ID":"f6615509-6990-4059-b3f3-4aad3cd7a366","Type":"ContainerDied","Data":"97eca87bf24d82a35bb6ecafa7c3c958f2da95fbbda7b44de7fef404e8852842"} Aug 04 11:10:30.001561 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:30.001545 2535 scope.go:117] "RemoveContainer" containerID="f18dae808341e8e10fd83041e7300917c02166c8d45bfd6b366159f9313399a2" Aug 04 11:10:30.001812 ip-10-0-130-250 kubenswrapper[2535]: E0804 11:10:30.001791 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f18dae808341e8e10fd83041e7300917c02166c8d45bfd6b366159f9313399a2\": container with ID starting with f18dae808341e8e10fd83041e7300917c02166c8d45bfd6b366159f9313399a2 not found: ID does not exist" containerID="f18dae808341e8e10fd83041e7300917c02166c8d45bfd6b366159f9313399a2" Aug 04 11:10:30.001864 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:30.001821 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f18dae808341e8e10fd83041e7300917c02166c8d45bfd6b366159f9313399a2"} err="failed to get container status \"f18dae808341e8e10fd83041e7300917c02166c8d45bfd6b366159f9313399a2\": rpc error: code = NotFound desc = could not find container \"f18dae808341e8e10fd83041e7300917c02166c8d45bfd6b366159f9313399a2\": container with ID starting with f18dae808341e8e10fd83041e7300917c02166c8d45bfd6b366159f9313399a2 not found: ID does not exist" Aug 04 11:10:30.001864 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:30.001838 2535 scope.go:117] "RemoveContainer" containerID="829c9b82abdaeb78caa3d20b3a565ad69e4a9a8b920803c3a2933b5b753064a1" Aug 04 11:10:30.009147 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:30.009126 2535 scope.go:117] "RemoveContainer" containerID="829c9b82abdaeb78caa3d20b3a565ad69e4a9a8b920803c3a2933b5b753064a1" Aug 04 11:10:30.009398 ip-10-0-130-250 kubenswrapper[2535]: E0804 11:10:30.009376 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"829c9b82abdaeb78caa3d20b3a565ad69e4a9a8b920803c3a2933b5b753064a1\": container with ID starting with 829c9b82abdaeb78caa3d20b3a565ad69e4a9a8b920803c3a2933b5b753064a1 not found: ID does not exist" containerID="829c9b82abdaeb78caa3d20b3a565ad69e4a9a8b920803c3a2933b5b753064a1" Aug 04 11:10:30.009452 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:30.009410 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"829c9b82abdaeb78caa3d20b3a565ad69e4a9a8b920803c3a2933b5b753064a1"} err="failed to get container status \"829c9b82abdaeb78caa3d20b3a565ad69e4a9a8b920803c3a2933b5b753064a1\": rpc error: code = NotFound desc = could not find container \"829c9b82abdaeb78caa3d20b3a565ad69e4a9a8b920803c3a2933b5b753064a1\": container with ID starting with 829c9b82abdaeb78caa3d20b3a565ad69e4a9a8b920803c3a2933b5b753064a1 not found: ID does not exist" Aug 04 11:10:30.041749 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:30.041718 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6615509-6990-4059-b3f3-4aad3cd7a366-plugins-config-volume\") pod \"f6615509-6990-4059-b3f3-4aad3cd7a366\" (UID: \"f6615509-6990-4059-b3f3-4aad3cd7a366\") " Aug 04 11:10:30.041917 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:30.041754 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e4b61c8-b17b-41dd-93df-e1e22f13a8cc-plugins-config-volume\") pod \"3e4b61c8-b17b-41dd-93df-e1e22f13a8cc\" (UID: \"3e4b61c8-b17b-41dd-93df-e1e22f13a8cc\") " Aug 04 11:10:30.041917 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:30.041846 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxs27\" (UniqueName: \"kubernetes.io/projected/f6615509-6990-4059-b3f3-4aad3cd7a366-kube-api-access-jxs27\") pod \"f6615509-6990-4059-b3f3-4aad3cd7a366\" (UID: \"f6615509-6990-4059-b3f3-4aad3cd7a366\") " Aug 04 11:10:30.041917 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:30.041882 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ztj2b\" (UniqueName: \"kubernetes.io/projected/3e4b61c8-b17b-41dd-93df-e1e22f13a8cc-kube-api-access-ztj2b\") pod \"3e4b61c8-b17b-41dd-93df-e1e22f13a8cc\" (UID: \"3e4b61c8-b17b-41dd-93df-e1e22f13a8cc\") " Aug 04 11:10:30.042151 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:30.042128 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e4b61c8-b17b-41dd-93df-e1e22f13a8cc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3e4b61c8-b17b-41dd-93df-e1e22f13a8cc" (UID: "3e4b61c8-b17b-41dd-93df-e1e22f13a8cc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:10:30.042197 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:30.042141 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6615509-6990-4059-b3f3-4aad3cd7a366-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f6615509-6990-4059-b3f3-4aad3cd7a366" (UID: "f6615509-6990-4059-b3f3-4aad3cd7a366"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:10:30.043843 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:30.043822 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6615509-6990-4059-b3f3-4aad3cd7a366-kube-api-access-jxs27" (OuterVolumeSpecName: "kube-api-access-jxs27") pod "f6615509-6990-4059-b3f3-4aad3cd7a366" (UID: "f6615509-6990-4059-b3f3-4aad3cd7a366"). InnerVolumeSpecName "kube-api-access-jxs27". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 11:10:30.043916 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:30.043901 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e4b61c8-b17b-41dd-93df-e1e22f13a8cc-kube-api-access-ztj2b" (OuterVolumeSpecName: "kube-api-access-ztj2b") pod "3e4b61c8-b17b-41dd-93df-e1e22f13a8cc" (UID: "3e4b61c8-b17b-41dd-93df-e1e22f13a8cc"). InnerVolumeSpecName "kube-api-access-ztj2b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 11:10:30.143015 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:30.142935 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxs27\" (UniqueName: \"kubernetes.io/projected/f6615509-6990-4059-b3f3-4aad3cd7a366-kube-api-access-jxs27\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:10:30.143015 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:30.142963 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztj2b\" (UniqueName: \"kubernetes.io/projected/3e4b61c8-b17b-41dd-93df-e1e22f13a8cc-kube-api-access-ztj2b\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:10:30.143015 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:30.142977 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6615509-6990-4059-b3f3-4aad3cd7a366-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:10:30.143015 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:30.142992 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e4b61c8-b17b-41dd-93df-e1e22f13a8cc-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:10:30.280569 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:30.280540 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:10:30.315178 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:30.315152 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c"] Aug 04 11:10:30.316880 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:30.316846 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-1c7c26-d9d5fb8bd-97v4c"] Aug 04 11:10:30.325098 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:30.325073 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn"] Aug 04 11:10:30.328216 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:30.328197 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-1c7c26-7847d9f4d8-xx8fn"] Aug 04 11:10:30.479275 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:30.479199 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e4b61c8-b17b-41dd-93df-e1e22f13a8cc" path="/var/lib/kubelet/pods/3e4b61c8-b17b-41dd-93df-e1e22f13a8cc/volumes" Aug 04 11:10:30.479602 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:30.479587 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6615509-6990-4059-b3f3-4aad3cd7a366" path="/var/lib/kubelet/pods/f6615509-6990-4059-b3f3-4aad3cd7a366/volumes" Aug 04 11:10:36.286326 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:36.286290 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:10:36.389870 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:36.389835 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:10:36.555629 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:36.555550 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:10:36.800280 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:36.800250 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:10:37.149447 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:37.149414 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:10:37.445502 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:37.445402 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:10:37.557020 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:37.556988 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:10:40.906820 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:40.906780 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:10:41.007986 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:41.007950 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:10:42.030253 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.030223 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:10:42.256203 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.256165 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:10:42.626768 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.626730 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6"] Aug 04 11:10:42.627086 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.627074 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e4b61c8-b17b-41dd-93df-e1e22f13a8cc" containerName="payload-processing" Aug 04 11:10:42.627135 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.627088 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e4b61c8-b17b-41dd-93df-e1e22f13a8cc" containerName="payload-processing" Aug 04 11:10:42.627135 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.627100 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26b81f19-f2e3-4727-ad02-163ec89e9673" containerName="payload-processing" Aug 04 11:10:42.627135 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.627105 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="26b81f19-f2e3-4727-ad02-163ec89e9673" containerName="payload-processing" Aug 04 11:10:42.627135 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.627123 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6615509-6990-4059-b3f3-4aad3cd7a366" containerName="payload-pre-processing" Aug 04 11:10:42.627135 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.627129 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6615509-6990-4059-b3f3-4aad3cd7a366" containerName="payload-pre-processing" Aug 04 11:10:42.627135 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.627137 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="060dfc25-87aa-4d07-98f7-ea62af1baea5" containerName="payload-pre-processing" Aug 04 11:10:42.627319 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.627143 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="060dfc25-87aa-4d07-98f7-ea62af1baea5" containerName="payload-pre-processing" Aug 04 11:10:42.627319 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.627190 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="26b81f19-f2e3-4727-ad02-163ec89e9673" containerName="payload-processing" Aug 04 11:10:42.627319 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.627200 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e4b61c8-b17b-41dd-93df-e1e22f13a8cc" containerName="payload-processing" Aug 04 11:10:42.627319 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.627206 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6615509-6990-4059-b3f3-4aad3cd7a366" containerName="payload-pre-processing" Aug 04 11:10:42.627319 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.627215 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="060dfc25-87aa-4d07-98f7-ea62af1baea5" containerName="payload-pre-processing" Aug 04 11:10:42.630350 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.630324 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6" Aug 04 11:10:42.632304 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.632278 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-1154a6-dockercfg-xczhb\"" Aug 04 11:10:42.632427 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.632326 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-1154a6\"" Aug 04 11:10:42.639087 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.638653 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6"] Aug 04 11:10:42.642161 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.642131 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlzq6\" (UniqueName: \"kubernetes.io/projected/0591fd02-f241-435b-b809-196a4de8e9b2-kube-api-access-dlzq6\") pod \"payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6\" (UID: \"0591fd02-f241-435b-b809-196a4de8e9b2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6" Aug 04 11:10:42.642350 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.642318 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0591fd02-f241-435b-b809-196a4de8e9b2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6\" (UID: \"0591fd02-f241-435b-b809-196a4de8e9b2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6" Aug 04 11:10:42.658883 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.658854 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m"] Aug 04 11:10:42.662282 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.662264 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m" Aug 04 11:10:42.673263 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.673237 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m"] Aug 04 11:10:42.743044 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.743010 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0591fd02-f241-435b-b809-196a4de8e9b2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6\" (UID: \"0591fd02-f241-435b-b809-196a4de8e9b2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6" Aug 04 11:10:42.743148 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.743076 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92dvk\" (UniqueName: \"kubernetes.io/projected/28314ff2-a5ac-4368-bf41-5e27ae3e8cdf-kube-api-access-92dvk\") pod \"payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m\" (UID: \"28314ff2-a5ac-4368-bf41-5e27ae3e8cdf\") " pod="openshift-ingress/payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m" Aug 04 11:10:42.743148 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.743129 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28314ff2-a5ac-4368-bf41-5e27ae3e8cdf-plugins-config-volume\") pod \"payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m\" (UID: \"28314ff2-a5ac-4368-bf41-5e27ae3e8cdf\") " pod="openshift-ingress/payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m" Aug 04 11:10:42.743261 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.743158 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlzq6\" (UniqueName: \"kubernetes.io/projected/0591fd02-f241-435b-b809-196a4de8e9b2-kube-api-access-dlzq6\") pod \"payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6\" (UID: \"0591fd02-f241-435b-b809-196a4de8e9b2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6" Aug 04 11:10:42.743904 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.743881 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0591fd02-f241-435b-b809-196a4de8e9b2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6\" (UID: \"0591fd02-f241-435b-b809-196a4de8e9b2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6" Aug 04 11:10:42.753083 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.753061 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlzq6\" (UniqueName: \"kubernetes.io/projected/0591fd02-f241-435b-b809-196a4de8e9b2-kube-api-access-dlzq6\") pod \"payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6\" (UID: \"0591fd02-f241-435b-b809-196a4de8e9b2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6" Aug 04 11:10:42.844019 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.843980 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92dvk\" (UniqueName: \"kubernetes.io/projected/28314ff2-a5ac-4368-bf41-5e27ae3e8cdf-kube-api-access-92dvk\") pod \"payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m\" (UID: \"28314ff2-a5ac-4368-bf41-5e27ae3e8cdf\") " pod="openshift-ingress/payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m" Aug 04 11:10:42.844158 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.844044 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28314ff2-a5ac-4368-bf41-5e27ae3e8cdf-plugins-config-volume\") pod \"payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m\" (UID: \"28314ff2-a5ac-4368-bf41-5e27ae3e8cdf\") " pod="openshift-ingress/payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m" Aug 04 11:10:42.844723 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.844705 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28314ff2-a5ac-4368-bf41-5e27ae3e8cdf-plugins-config-volume\") pod \"payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m\" (UID: \"28314ff2-a5ac-4368-bf41-5e27ae3e8cdf\") " pod="openshift-ingress/payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m" Aug 04 11:10:42.851591 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.851570 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92dvk\" (UniqueName: \"kubernetes.io/projected/28314ff2-a5ac-4368-bf41-5e27ae3e8cdf-kube-api-access-92dvk\") pod \"payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m\" (UID: \"28314ff2-a5ac-4368-bf41-5e27ae3e8cdf\") " pod="openshift-ingress/payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m" Aug 04 11:10:42.943415 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.943359 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6" Aug 04 11:10:42.972413 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:42.972374 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m" Aug 04 11:10:43.121147 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:43.120787 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m"] Aug 04 11:10:43.123445 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:10:43.123412 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28314ff2_a5ac_4368_bf41_5e27ae3e8cdf.slice/crio-5a8ce89df94e1990b2821386bdcfda51f844f14fe4c548c477eeb644e461c2f9 WatchSource:0}: Error finding container 5a8ce89df94e1990b2821386bdcfda51f844f14fe4c548c477eeb644e461c2f9: Status 404 returned error can't find the container with id 5a8ce89df94e1990b2821386bdcfda51f844f14fe4c548c477eeb644e461c2f9 Aug 04 11:10:43.139979 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:43.139951 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:10:43.240965 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:43.240935 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:10:43.300170 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:43.300144 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6"] Aug 04 11:10:43.301794 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:10:43.301764 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0591fd02_f241_435b_b809_196a4de8e9b2.slice/crio-501eae3a97ebda67c00f704c08cfefb7d2457e031d30e1910124509f3711b32c WatchSource:0}: Error finding container 501eae3a97ebda67c00f704c08cfefb7d2457e031d30e1910124509f3711b32c: Status 404 returned error can't find the container with id 501eae3a97ebda67c00f704c08cfefb7d2457e031d30e1910124509f3711b32c Aug 04 11:10:44.062881 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:44.062844 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6" event={"ID":"0591fd02-f241-435b-b809-196a4de8e9b2","Type":"ContainerStarted","Data":"b642af2c50cc41564e5e0caeb2e7f6b2050c9c0b1bf44a57f7a2b83550e79ca8"} Aug 04 11:10:44.063088 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:44.062906 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6" event={"ID":"0591fd02-f241-435b-b809-196a4de8e9b2","Type":"ContainerStarted","Data":"501eae3a97ebda67c00f704c08cfefb7d2457e031d30e1910124509f3711b32c"} Aug 04 11:10:44.063088 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:44.063055 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6" Aug 04 11:10:44.064311 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:44.064284 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m" event={"ID":"28314ff2-a5ac-4368-bf41-5e27ae3e8cdf","Type":"ContainerStarted","Data":"55c97da134605ed5af1027a3a8e98356afc475681da893da718ce2a9e4eb93c3"} Aug 04 11:10:44.064451 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:44.064328 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m" event={"ID":"28314ff2-a5ac-4368-bf41-5e27ae3e8cdf","Type":"ContainerStarted","Data":"5a8ce89df94e1990b2821386bdcfda51f844f14fe4c548c477eeb644e461c2f9"} Aug 04 11:10:44.064451 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:44.064366 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m" Aug 04 11:10:44.080090 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:44.080046 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6" podStartSLOduration=2.080032936 podStartE2EDuration="2.080032936s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:10:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:10:44.078082102 +0000 UTC m=+2582.052164644" watchObservedRunningTime="2026-08-04 11:10:44.080032936 +0000 UTC m=+2582.054115463" Aug 04 11:10:44.094570 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:44.094525 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m" podStartSLOduration=2.094511553 podStartE2EDuration="2.094511553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:10:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:10:44.092950263 +0000 UTC m=+2582.067032793" watchObservedRunningTime="2026-08-04 11:10:44.094511553 +0000 UTC m=+2582.068594080" Aug 04 11:10:53.730733 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:53.730692 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:10:55.071084 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:55.071055 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m" Aug 04 11:10:55.071452 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:55.071123 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6" Aug 04 11:10:57.141949 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:10:57.141915 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:11:05.943006 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:05.942929 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:11:06.382281 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:06.382243 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:11:06.696323 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:06.696242 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:11:06.931250 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:06.931214 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:11:10.054670 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:10.054638 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:11:10.161102 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:10.161071 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:11:11.155581 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:11.155547 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:11:11.781268 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:11.781236 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:11:11.880948 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:11.880915 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:11:11.887055 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:11.887031 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9"] Aug 04 11:11:11.890808 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:11.890791 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9" Aug 04 11:11:11.892925 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:11.892906 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-74c752\"" Aug 04 11:11:11.893015 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:11.892966 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-74c752-dockercfg-85867\"" Aug 04 11:11:11.901750 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:11.901726 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9"] Aug 04 11:11:11.928466 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:11.928431 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58"] Aug 04 11:11:11.932001 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:11.931982 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58" Aug 04 11:11:11.948201 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:11.947023 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58"] Aug 04 11:11:12.020807 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:12.020768 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbql2\" (UniqueName: \"kubernetes.io/projected/92c055d5-7ba4-4671-84a3-aeec50b1720d-kube-api-access-hbql2\") pod \"payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58\" (UID: \"92c055d5-7ba4-4671-84a3-aeec50b1720d\") " pod="openshift-ingress/payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58" Aug 04 11:11:12.021008 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:12.020846 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92c055d5-7ba4-4671-84a3-aeec50b1720d-plugins-config-volume\") pod \"payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58\" (UID: \"92c055d5-7ba4-4671-84a3-aeec50b1720d\") " pod="openshift-ingress/payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58" Aug 04 11:11:12.021008 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:12.020880 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af980e23-b1cf-49ac-b4c7-372e598842a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9\" (UID: \"af980e23-b1cf-49ac-b4c7-372e598842a2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9" Aug 04 11:11:12.021008 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:12.020956 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq954\" (UniqueName: \"kubernetes.io/projected/af980e23-b1cf-49ac-b4c7-372e598842a2-kube-api-access-nq954\") pod \"payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9\" (UID: \"af980e23-b1cf-49ac-b4c7-372e598842a2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9" Aug 04 11:11:12.122012 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:12.121919 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq954\" (UniqueName: \"kubernetes.io/projected/af980e23-b1cf-49ac-b4c7-372e598842a2-kube-api-access-nq954\") pod \"payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9\" (UID: \"af980e23-b1cf-49ac-b4c7-372e598842a2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9" Aug 04 11:11:12.122012 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:12.122004 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbql2\" (UniqueName: \"kubernetes.io/projected/92c055d5-7ba4-4671-84a3-aeec50b1720d-kube-api-access-hbql2\") pod \"payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58\" (UID: \"92c055d5-7ba4-4671-84a3-aeec50b1720d\") " pod="openshift-ingress/payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58" Aug 04 11:11:12.122252 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:12.122038 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92c055d5-7ba4-4671-84a3-aeec50b1720d-plugins-config-volume\") pod \"payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58\" (UID: \"92c055d5-7ba4-4671-84a3-aeec50b1720d\") " pod="openshift-ingress/payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58" Aug 04 11:11:12.122252 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:12.122058 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af980e23-b1cf-49ac-b4c7-372e598842a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9\" (UID: \"af980e23-b1cf-49ac-b4c7-372e598842a2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9" Aug 04 11:11:12.122758 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:12.122740 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af980e23-b1cf-49ac-b4c7-372e598842a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9\" (UID: \"af980e23-b1cf-49ac-b4c7-372e598842a2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9" Aug 04 11:11:12.122809 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:12.122753 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92c055d5-7ba4-4671-84a3-aeec50b1720d-plugins-config-volume\") pod \"payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58\" (UID: \"92c055d5-7ba4-4671-84a3-aeec50b1720d\") " pod="openshift-ingress/payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58" Aug 04 11:11:12.130547 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:12.130525 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq954\" (UniqueName: \"kubernetes.io/projected/af980e23-b1cf-49ac-b4c7-372e598842a2-kube-api-access-nq954\") pod \"payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9\" (UID: \"af980e23-b1cf-49ac-b4c7-372e598842a2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9" Aug 04 11:11:12.130745 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:12.130725 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbql2\" (UniqueName: \"kubernetes.io/projected/92c055d5-7ba4-4671-84a3-aeec50b1720d-kube-api-access-hbql2\") pod \"payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58\" (UID: \"92c055d5-7ba4-4671-84a3-aeec50b1720d\") " pod="openshift-ingress/payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58" Aug 04 11:11:12.202549 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:12.202518 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9" Aug 04 11:11:12.241204 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:12.241147 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58" Aug 04 11:11:12.346569 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:12.346536 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9"] Aug 04 11:11:12.349835 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:11:12.349809 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf980e23_b1cf_49ac_b4c7_372e598842a2.slice/crio-d840a62d7b7770d60fc1f07c3f9c1b663d3380e2d8bbb8b714e5eeac75b46c21 WatchSource:0}: Error finding container d840a62d7b7770d60fc1f07c3f9c1b663d3380e2d8bbb8b714e5eeac75b46c21: Status 404 returned error can't find the container with id d840a62d7b7770d60fc1f07c3f9c1b663d3380e2d8bbb8b714e5eeac75b46c21 Aug 04 11:11:12.390764 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:12.390736 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58"] Aug 04 11:11:12.405579 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:11:12.405553 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92c055d5_7ba4_4671_84a3_aeec50b1720d.slice/crio-f9edc4109760a751ef50d33885f38bceffb8efb984188b46e2ec4c45d65c1b72 WatchSource:0}: Error finding container f9edc4109760a751ef50d33885f38bceffb8efb984188b46e2ec4c45d65c1b72: Status 404 returned error can't find the container with id f9edc4109760a751ef50d33885f38bceffb8efb984188b46e2ec4c45d65c1b72 Aug 04 11:11:13.215005 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:13.214962 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9" event={"ID":"af980e23-b1cf-49ac-b4c7-372e598842a2","Type":"ContainerStarted","Data":"c96bf5c065c89ce34b4356d91e018967a27d64b0a153a81089e61d333c653c10"} Aug 04 11:11:13.215525 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:13.215050 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9" event={"ID":"af980e23-b1cf-49ac-b4c7-372e598842a2","Type":"ContainerStarted","Data":"d840a62d7b7770d60fc1f07c3f9c1b663d3380e2d8bbb8b714e5eeac75b46c21"} Aug 04 11:11:13.215525 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:13.215150 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9" Aug 04 11:11:13.216647 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:13.216624 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58" event={"ID":"92c055d5-7ba4-4671-84a3-aeec50b1720d","Type":"ContainerStarted","Data":"2f9af1faa3cf2573bf7fa7f3950eb42a473dd56ef0b6c7733815dffaff16e476"} Aug 04 11:11:13.216732 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:13.216666 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58" Aug 04 11:11:13.216732 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:13.216681 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58" event={"ID":"92c055d5-7ba4-4671-84a3-aeec50b1720d","Type":"ContainerStarted","Data":"f9edc4109760a751ef50d33885f38bceffb8efb984188b46e2ec4c45d65c1b72"} Aug 04 11:11:13.230923 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:13.230710 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9" podStartSLOduration=2.230695598 podStartE2EDuration="2.230695598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:11:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:11:13.23032206 +0000 UTC m=+2611.204404614" watchObservedRunningTime="2026-08-04 11:11:13.230695598 +0000 UTC m=+2611.204778122" Aug 04 11:11:13.245171 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:13.245125 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58" podStartSLOduration=2.245108081 podStartE2EDuration="2.245108081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:11:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:11:13.244836369 +0000 UTC m=+2611.218918925" watchObservedRunningTime="2026-08-04 11:11:13.245108081 +0000 UTC m=+2611.219190608" Aug 04 11:11:24.223820 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:24.223762 2535 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9" podUID="af980e23-b1cf-49ac-b4c7-372e598842a2" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.55:9004: connect: connection refused" Aug 04 11:11:24.226261 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:24.224305 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58" Aug 04 11:11:34.222063 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:34.222033 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9" Aug 04 11:11:40.206252 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:40.206221 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:11:58.736548 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:58.736514 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:11:58.744626 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:58.744595 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m"] Aug 04 11:11:58.744887 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:58.744862 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m" podUID="28314ff2-a5ac-4368-bf41-5e27ae3e8cdf" containerName="payload-processing" containerID="cri-o://55c97da134605ed5af1027a3a8e98356afc475681da893da718ce2a9e4eb93c3" gracePeriod=30 Aug 04 11:11:58.752129 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:58.752100 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6"] Aug 04 11:11:58.752356 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:58.752329 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6" podUID="0591fd02-f241-435b-b809-196a4de8e9b2" containerName="payload-pre-processing" containerID="cri-o://b642af2c50cc41564e5e0caeb2e7f6b2050c9c0b1bf44a57f7a2b83550e79ca8" gracePeriod=30 Aug 04 11:11:58.831029 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:58.831001 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:11:58.858781 ip-10-0-130-250 kubenswrapper[2535]: E0804 11:11:58.858750 2535 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0591fd02_f241_435b_b809_196a4de8e9b2.slice/crio-b642af2c50cc41564e5e0caeb2e7f6b2050c9c0b1bf44a57f7a2b83550e79ca8.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0591fd02_f241_435b_b809_196a4de8e9b2.slice/crio-conmon-b642af2c50cc41564e5e0caeb2e7f6b2050c9c0b1bf44a57f7a2b83550e79ca8.scope\": RecentStats: unable to find data in memory cache]" Aug 04 11:11:58.899222 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:58.899125 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:11:59.029706 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.029675 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m" Aug 04 11:11:59.032679 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.032661 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6" Aug 04 11:11:59.057125 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.057103 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92dvk\" (UniqueName: \"kubernetes.io/projected/28314ff2-a5ac-4368-bf41-5e27ae3e8cdf-kube-api-access-92dvk\") pod \"28314ff2-a5ac-4368-bf41-5e27ae3e8cdf\" (UID: \"28314ff2-a5ac-4368-bf41-5e27ae3e8cdf\") " Aug 04 11:11:59.057237 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.057152 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0591fd02-f241-435b-b809-196a4de8e9b2-plugins-config-volume\") pod \"0591fd02-f241-435b-b809-196a4de8e9b2\" (UID: \"0591fd02-f241-435b-b809-196a4de8e9b2\") " Aug 04 11:11:59.057346 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.057328 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dlzq6\" (UniqueName: \"kubernetes.io/projected/0591fd02-f241-435b-b809-196a4de8e9b2-kube-api-access-dlzq6\") pod \"0591fd02-f241-435b-b809-196a4de8e9b2\" (UID: \"0591fd02-f241-435b-b809-196a4de8e9b2\") " Aug 04 11:11:59.057398 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.057362 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28314ff2-a5ac-4368-bf41-5e27ae3e8cdf-plugins-config-volume\") pod \"28314ff2-a5ac-4368-bf41-5e27ae3e8cdf\" (UID: \"28314ff2-a5ac-4368-bf41-5e27ae3e8cdf\") " Aug 04 11:11:59.057547 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.057523 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0591fd02-f241-435b-b809-196a4de8e9b2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0591fd02-f241-435b-b809-196a4de8e9b2" (UID: "0591fd02-f241-435b-b809-196a4de8e9b2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:11:59.057781 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.057764 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0591fd02-f241-435b-b809-196a4de8e9b2-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:11:59.057846 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.057810 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28314ff2-a5ac-4368-bf41-5e27ae3e8cdf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "28314ff2-a5ac-4368-bf41-5e27ae3e8cdf" (UID: "28314ff2-a5ac-4368-bf41-5e27ae3e8cdf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:11:59.059210 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.059184 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28314ff2-a5ac-4368-bf41-5e27ae3e8cdf-kube-api-access-92dvk" (OuterVolumeSpecName: "kube-api-access-92dvk") pod "28314ff2-a5ac-4368-bf41-5e27ae3e8cdf" (UID: "28314ff2-a5ac-4368-bf41-5e27ae3e8cdf"). InnerVolumeSpecName "kube-api-access-92dvk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 11:11:59.059210 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.059198 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0591fd02-f241-435b-b809-196a4de8e9b2-kube-api-access-dlzq6" (OuterVolumeSpecName: "kube-api-access-dlzq6") pod "0591fd02-f241-435b-b809-196a4de8e9b2" (UID: "0591fd02-f241-435b-b809-196a4de8e9b2"). InnerVolumeSpecName "kube-api-access-dlzq6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 11:11:59.158337 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.158305 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlzq6\" (UniqueName: \"kubernetes.io/projected/0591fd02-f241-435b-b809-196a4de8e9b2-kube-api-access-dlzq6\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:11:59.158337 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.158334 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28314ff2-a5ac-4368-bf41-5e27ae3e8cdf-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:11:59.158337 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.158344 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92dvk\" (UniqueName: \"kubernetes.io/projected/28314ff2-a5ac-4368-bf41-5e27ae3e8cdf-kube-api-access-92dvk\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:11:59.265674 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.265589 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:11:59.437214 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.437179 2535 generic.go:356] "Generic (PLEG): container finished" podID="0591fd02-f241-435b-b809-196a4de8e9b2" containerID="b642af2c50cc41564e5e0caeb2e7f6b2050c9c0b1bf44a57f7a2b83550e79ca8" exitCode=0 Aug 04 11:11:59.437399 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.437221 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6" Aug 04 11:11:59.437399 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.437268 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6" event={"ID":"0591fd02-f241-435b-b809-196a4de8e9b2","Type":"ContainerDied","Data":"b642af2c50cc41564e5e0caeb2e7f6b2050c9c0b1bf44a57f7a2b83550e79ca8"} Aug 04 11:11:59.437399 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.437330 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6" event={"ID":"0591fd02-f241-435b-b809-196a4de8e9b2","Type":"ContainerDied","Data":"501eae3a97ebda67c00f704c08cfefb7d2457e031d30e1910124509f3711b32c"} Aug 04 11:11:59.437399 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.437360 2535 scope.go:117] "RemoveContainer" containerID="b642af2c50cc41564e5e0caeb2e7f6b2050c9c0b1bf44a57f7a2b83550e79ca8" Aug 04 11:11:59.438669 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.438557 2535 generic.go:356] "Generic (PLEG): container finished" podID="28314ff2-a5ac-4368-bf41-5e27ae3e8cdf" containerID="55c97da134605ed5af1027a3a8e98356afc475681da893da718ce2a9e4eb93c3" exitCode=0 Aug 04 11:11:59.438669 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.438598 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m" event={"ID":"28314ff2-a5ac-4368-bf41-5e27ae3e8cdf","Type":"ContainerDied","Data":"55c97da134605ed5af1027a3a8e98356afc475681da893da718ce2a9e4eb93c3"} Aug 04 11:11:59.438669 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.438616 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m" Aug 04 11:11:59.438669 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.438635 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m" event={"ID":"28314ff2-a5ac-4368-bf41-5e27ae3e8cdf","Type":"ContainerDied","Data":"5a8ce89df94e1990b2821386bdcfda51f844f14fe4c548c477eeb644e461c2f9"} Aug 04 11:11:59.446529 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.446430 2535 scope.go:117] "RemoveContainer" containerID="b642af2c50cc41564e5e0caeb2e7f6b2050c9c0b1bf44a57f7a2b83550e79ca8" Aug 04 11:11:59.446844 ip-10-0-130-250 kubenswrapper[2535]: E0804 11:11:59.446815 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b642af2c50cc41564e5e0caeb2e7f6b2050c9c0b1bf44a57f7a2b83550e79ca8\": container with ID starting with b642af2c50cc41564e5e0caeb2e7f6b2050c9c0b1bf44a57f7a2b83550e79ca8 not found: ID does not exist" containerID="b642af2c50cc41564e5e0caeb2e7f6b2050c9c0b1bf44a57f7a2b83550e79ca8" Aug 04 11:11:59.446940 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.446852 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b642af2c50cc41564e5e0caeb2e7f6b2050c9c0b1bf44a57f7a2b83550e79ca8"} err="failed to get container status \"b642af2c50cc41564e5e0caeb2e7f6b2050c9c0b1bf44a57f7a2b83550e79ca8\": rpc error: code = NotFound desc = could not find container \"b642af2c50cc41564e5e0caeb2e7f6b2050c9c0b1bf44a57f7a2b83550e79ca8\": container with ID starting with b642af2c50cc41564e5e0caeb2e7f6b2050c9c0b1bf44a57f7a2b83550e79ca8 not found: ID does not exist" Aug 04 11:11:59.446940 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.446868 2535 scope.go:117] "RemoveContainer" containerID="55c97da134605ed5af1027a3a8e98356afc475681da893da718ce2a9e4eb93c3" Aug 04 11:11:59.454724 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.454708 2535 scope.go:117] "RemoveContainer" containerID="55c97da134605ed5af1027a3a8e98356afc475681da893da718ce2a9e4eb93c3" Aug 04 11:11:59.454999 ip-10-0-130-250 kubenswrapper[2535]: E0804 11:11:59.454979 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55c97da134605ed5af1027a3a8e98356afc475681da893da718ce2a9e4eb93c3\": container with ID starting with 55c97da134605ed5af1027a3a8e98356afc475681da893da718ce2a9e4eb93c3 not found: ID does not exist" containerID="55c97da134605ed5af1027a3a8e98356afc475681da893da718ce2a9e4eb93c3" Aug 04 11:11:59.455056 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.455005 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55c97da134605ed5af1027a3a8e98356afc475681da893da718ce2a9e4eb93c3"} err="failed to get container status \"55c97da134605ed5af1027a3a8e98356afc475681da893da718ce2a9e4eb93c3\": rpc error: code = NotFound desc = could not find container \"55c97da134605ed5af1027a3a8e98356afc475681da893da718ce2a9e4eb93c3\": container with ID starting with 55c97da134605ed5af1027a3a8e98356afc475681da893da718ce2a9e4eb93c3 not found: ID does not exist" Aug 04 11:11:59.463102 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.463079 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6"] Aug 04 11:11:59.466128 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.466103 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1154a6-6dbc685f84-bnst6"] Aug 04 11:11:59.476356 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.476329 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m"] Aug 04 11:11:59.479670 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:11:59.479650 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1154a6-7dc66b549b-cgx5m"] Aug 04 11:12:00.479825 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:00.479787 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0591fd02-f241-435b-b809-196a4de8e9b2" path="/var/lib/kubelet/pods/0591fd02-f241-435b-b809-196a4de8e9b2/volumes" Aug 04 11:12:00.480303 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:00.480280 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28314ff2-a5ac-4368-bf41-5e27ae3e8cdf" path="/var/lib/kubelet/pods/28314ff2-a5ac-4368-bf41-5e27ae3e8cdf/volumes" Aug 04 11:12:05.373922 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:05.373887 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:12:05.444681 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:05.444652 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:12:05.634017 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:05.633940 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:12:05.867010 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:05.866979 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:12:21.537409 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:21.537365 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:12:21.568153 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:21.568120 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58"] Aug 04 11:12:21.568410 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:21.568384 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58" podUID="92c055d5-7ba4-4671-84a3-aeec50b1720d" containerName="payload-processing" containerID="cri-o://2f9af1faa3cf2573bf7fa7f3950eb42a473dd56ef0b6c7733815dffaff16e476" gracePeriod=30 Aug 04 11:12:21.575931 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:21.575905 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9"] Aug 04 11:12:21.576165 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:21.576127 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9" podUID="af980e23-b1cf-49ac-b4c7-372e598842a2" containerName="payload-pre-processing" containerID="cri-o://c96bf5c065c89ce34b4356d91e018967a27d64b0a153a81089e61d333c653c10" gracePeriod=30 Aug 04 11:12:21.602034 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:21.602008 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:12:21.678053 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:21.678024 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:12:21.778612 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:21.778496 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:12:21.840603 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:21.840580 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58" Aug 04 11:12:21.856212 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:21.856195 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9" Aug 04 11:12:21.964540 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:21.964504 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af980e23-b1cf-49ac-b4c7-372e598842a2-plugins-config-volume\") pod \"af980e23-b1cf-49ac-b4c7-372e598842a2\" (UID: \"af980e23-b1cf-49ac-b4c7-372e598842a2\") " Aug 04 11:12:21.964540 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:21.964547 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbql2\" (UniqueName: \"kubernetes.io/projected/92c055d5-7ba4-4671-84a3-aeec50b1720d-kube-api-access-hbql2\") pod \"92c055d5-7ba4-4671-84a3-aeec50b1720d\" (UID: \"92c055d5-7ba4-4671-84a3-aeec50b1720d\") " Aug 04 11:12:21.964766 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:21.964605 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nq954\" (UniqueName: \"kubernetes.io/projected/af980e23-b1cf-49ac-b4c7-372e598842a2-kube-api-access-nq954\") pod \"af980e23-b1cf-49ac-b4c7-372e598842a2\" (UID: \"af980e23-b1cf-49ac-b4c7-372e598842a2\") " Aug 04 11:12:21.964766 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:21.964636 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92c055d5-7ba4-4671-84a3-aeec50b1720d-plugins-config-volume\") pod \"92c055d5-7ba4-4671-84a3-aeec50b1720d\" (UID: \"92c055d5-7ba4-4671-84a3-aeec50b1720d\") " Aug 04 11:12:21.965004 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:21.964858 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af980e23-b1cf-49ac-b4c7-372e598842a2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "af980e23-b1cf-49ac-b4c7-372e598842a2" (UID: "af980e23-b1cf-49ac-b4c7-372e598842a2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:12:21.965072 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:21.965020 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92c055d5-7ba4-4671-84a3-aeec50b1720d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "92c055d5-7ba4-4671-84a3-aeec50b1720d" (UID: "92c055d5-7ba4-4671-84a3-aeec50b1720d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:12:21.966682 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:21.966661 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92c055d5-7ba4-4671-84a3-aeec50b1720d-kube-api-access-hbql2" (OuterVolumeSpecName: "kube-api-access-hbql2") pod "92c055d5-7ba4-4671-84a3-aeec50b1720d" (UID: "92c055d5-7ba4-4671-84a3-aeec50b1720d"). InnerVolumeSpecName "kube-api-access-hbql2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 11:12:21.966780 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:21.966691 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af980e23-b1cf-49ac-b4c7-372e598842a2-kube-api-access-nq954" (OuterVolumeSpecName: "kube-api-access-nq954") pod "af980e23-b1cf-49ac-b4c7-372e598842a2" (UID: "af980e23-b1cf-49ac-b4c7-372e598842a2"). InnerVolumeSpecName "kube-api-access-nq954". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 11:12:22.065396 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:22.065333 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nq954\" (UniqueName: \"kubernetes.io/projected/af980e23-b1cf-49ac-b4c7-372e598842a2-kube-api-access-nq954\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:12:22.065396 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:22.065356 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92c055d5-7ba4-4671-84a3-aeec50b1720d-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:12:22.065396 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:22.065367 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af980e23-b1cf-49ac-b4c7-372e598842a2-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:12:22.065396 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:22.065376 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbql2\" (UniqueName: \"kubernetes.io/projected/92c055d5-7ba4-4671-84a3-aeec50b1720d-kube-api-access-hbql2\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:12:22.134877 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:22.134848 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:12:22.552446 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:22.552414 2535 generic.go:356] "Generic (PLEG): container finished" podID="92c055d5-7ba4-4671-84a3-aeec50b1720d" containerID="2f9af1faa3cf2573bf7fa7f3950eb42a473dd56ef0b6c7733815dffaff16e476" exitCode=0 Aug 04 11:12:22.552841 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:22.552452 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58" Aug 04 11:12:22.552841 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:22.552508 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58" event={"ID":"92c055d5-7ba4-4671-84a3-aeec50b1720d","Type":"ContainerDied","Data":"2f9af1faa3cf2573bf7fa7f3950eb42a473dd56ef0b6c7733815dffaff16e476"} Aug 04 11:12:22.552841 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:22.552553 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58" event={"ID":"92c055d5-7ba4-4671-84a3-aeec50b1720d","Type":"ContainerDied","Data":"f9edc4109760a751ef50d33885f38bceffb8efb984188b46e2ec4c45d65c1b72"} Aug 04 11:12:22.552841 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:22.552575 2535 scope.go:117] "RemoveContainer" containerID="2f9af1faa3cf2573bf7fa7f3950eb42a473dd56ef0b6c7733815dffaff16e476" Aug 04 11:12:22.553861 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:22.553841 2535 generic.go:356] "Generic (PLEG): container finished" podID="af980e23-b1cf-49ac-b4c7-372e598842a2" containerID="c96bf5c065c89ce34b4356d91e018967a27d64b0a153a81089e61d333c653c10" exitCode=0 Aug 04 11:12:22.553956 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:22.553879 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9" Aug 04 11:12:22.553956 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:22.553890 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9" event={"ID":"af980e23-b1cf-49ac-b4c7-372e598842a2","Type":"ContainerDied","Data":"c96bf5c065c89ce34b4356d91e018967a27d64b0a153a81089e61d333c653c10"} Aug 04 11:12:22.553956 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:22.553944 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9" event={"ID":"af980e23-b1cf-49ac-b4c7-372e598842a2","Type":"ContainerDied","Data":"d840a62d7b7770d60fc1f07c3f9c1b663d3380e2d8bbb8b714e5eeac75b46c21"} Aug 04 11:12:22.561335 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:22.561317 2535 scope.go:117] "RemoveContainer" containerID="2f9af1faa3cf2573bf7fa7f3950eb42a473dd56ef0b6c7733815dffaff16e476" Aug 04 11:12:22.561609 ip-10-0-130-250 kubenswrapper[2535]: E0804 11:12:22.561589 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f9af1faa3cf2573bf7fa7f3950eb42a473dd56ef0b6c7733815dffaff16e476\": container with ID starting with 2f9af1faa3cf2573bf7fa7f3950eb42a473dd56ef0b6c7733815dffaff16e476 not found: ID does not exist" containerID="2f9af1faa3cf2573bf7fa7f3950eb42a473dd56ef0b6c7733815dffaff16e476" Aug 04 11:12:22.561702 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:22.561614 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f9af1faa3cf2573bf7fa7f3950eb42a473dd56ef0b6c7733815dffaff16e476"} err="failed to get container status \"2f9af1faa3cf2573bf7fa7f3950eb42a473dd56ef0b6c7733815dffaff16e476\": rpc error: code = NotFound desc = could not find container \"2f9af1faa3cf2573bf7fa7f3950eb42a473dd56ef0b6c7733815dffaff16e476\": container with ID starting with 2f9af1faa3cf2573bf7fa7f3950eb42a473dd56ef0b6c7733815dffaff16e476 not found: ID does not exist" Aug 04 11:12:22.561702 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:22.561630 2535 scope.go:117] "RemoveContainer" containerID="c96bf5c065c89ce34b4356d91e018967a27d64b0a153a81089e61d333c653c10" Aug 04 11:12:22.568787 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:22.568772 2535 scope.go:117] "RemoveContainer" containerID="c96bf5c065c89ce34b4356d91e018967a27d64b0a153a81089e61d333c653c10" Aug 04 11:12:22.569034 ip-10-0-130-250 kubenswrapper[2535]: E0804 11:12:22.569015 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c96bf5c065c89ce34b4356d91e018967a27d64b0a153a81089e61d333c653c10\": container with ID starting with c96bf5c065c89ce34b4356d91e018967a27d64b0a153a81089e61d333c653c10 not found: ID does not exist" containerID="c96bf5c065c89ce34b4356d91e018967a27d64b0a153a81089e61d333c653c10" Aug 04 11:12:22.569085 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:22.569042 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c96bf5c065c89ce34b4356d91e018967a27d64b0a153a81089e61d333c653c10"} err="failed to get container status \"c96bf5c065c89ce34b4356d91e018967a27d64b0a153a81089e61d333c653c10\": rpc error: code = NotFound desc = could not find container \"c96bf5c065c89ce34b4356d91e018967a27d64b0a153a81089e61d333c653c10\": container with ID starting with c96bf5c065c89ce34b4356d91e018967a27d64b0a153a81089e61d333c653c10 not found: ID does not exist" Aug 04 11:12:22.573034 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:22.573013 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58"] Aug 04 11:12:22.577217 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:22.577199 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-74c752-77df98cfd6-mxn58"] Aug 04 11:12:22.587823 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:22.587804 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9"] Aug 04 11:12:22.591444 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:22.591422 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-74c752-85478fc784-nzpp9"] Aug 04 11:12:24.479431 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:24.479397 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92c055d5-7ba4-4671-84a3-aeec50b1720d" path="/var/lib/kubelet/pods/92c055d5-7ba4-4671-84a3-aeec50b1720d/volumes" Aug 04 11:12:24.479856 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:24.479840 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af980e23-b1cf-49ac-b4c7-372e598842a2" path="/var/lib/kubelet/pods/af980e23-b1cf-49ac-b4c7-372e598842a2/volumes" Aug 04 11:12:28.344722 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:28.344691 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:12:28.444730 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:28.444698 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:12:28.617513 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:28.617422 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:12:28.855997 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:28.855960 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:12:29.202616 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:29.202583 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:12:29.554439 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:29.554407 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:12:29.851757 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:29.851663 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:12:32.460071 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:32.460034 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:12:32.590995 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:32.590963 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:12:33.653096 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:33.653041 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:12:33.759879 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:33.759842 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:12:34.230145 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.230115 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:12:34.302068 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.302036 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m"] Aug 04 11:12:34.302412 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.302400 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92c055d5-7ba4-4671-84a3-aeec50b1720d" containerName="payload-processing" Aug 04 11:12:34.302412 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.302414 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="92c055d5-7ba4-4671-84a3-aeec50b1720d" containerName="payload-processing" Aug 04 11:12:34.302521 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.302430 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28314ff2-a5ac-4368-bf41-5e27ae3e8cdf" containerName="payload-processing" Aug 04 11:12:34.302521 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.302435 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="28314ff2-a5ac-4368-bf41-5e27ae3e8cdf" containerName="payload-processing" Aug 04 11:12:34.302521 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.302443 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0591fd02-f241-435b-b809-196a4de8e9b2" containerName="payload-pre-processing" Aug 04 11:12:34.302521 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.302449 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="0591fd02-f241-435b-b809-196a4de8e9b2" containerName="payload-pre-processing" Aug 04 11:12:34.302521 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.302456 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af980e23-b1cf-49ac-b4c7-372e598842a2" containerName="payload-pre-processing" Aug 04 11:12:34.302521 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.302461 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="af980e23-b1cf-49ac-b4c7-372e598842a2" containerName="payload-pre-processing" Aug 04 11:12:34.302705 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.302531 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="0591fd02-f241-435b-b809-196a4de8e9b2" containerName="payload-pre-processing" Aug 04 11:12:34.302705 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.302540 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="af980e23-b1cf-49ac-b4c7-372e598842a2" containerName="payload-pre-processing" Aug 04 11:12:34.302705 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.302546 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="28314ff2-a5ac-4368-bf41-5e27ae3e8cdf" containerName="payload-processing" Aug 04 11:12:34.302705 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.302553 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="92c055d5-7ba4-4671-84a3-aeec50b1720d" containerName="payload-processing" Aug 04 11:12:34.307229 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.307213 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m" Aug 04 11:12:34.309333 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.309315 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-bc4a78\"" Aug 04 11:12:34.316085 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.316057 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m"] Aug 04 11:12:34.339544 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.339515 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:12:34.348456 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.348430 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb"] Aug 04 11:12:34.351798 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.351783 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb" Aug 04 11:12:34.353744 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.353724 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-bc4a78-dockercfg-chq8q\"" Aug 04 11:12:34.367067 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.367046 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb"] Aug 04 11:12:34.475855 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.475807 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l7kg\" (UniqueName: \"kubernetes.io/projected/a3c410c4-0152-44b8-995c-75bb6e1d6f16-kube-api-access-8l7kg\") pod \"payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb\" (UID: \"a3c410c4-0152-44b8-995c-75bb6e1d6f16\") " pod="openshift-ingress/payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb" Aug 04 11:12:34.475855 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.475861 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47026a7c-8e2c-4c7e-ac69-4e027cc868aa-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m\" (UID: \"47026a7c-8e2c-4c7e-ac69-4e027cc868aa\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m" Aug 04 11:12:34.476112 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.475880 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkwjt\" (UniqueName: \"kubernetes.io/projected/47026a7c-8e2c-4c7e-ac69-4e027cc868aa-kube-api-access-xkwjt\") pod \"payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m\" (UID: \"47026a7c-8e2c-4c7e-ac69-4e027cc868aa\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m" Aug 04 11:12:34.476112 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.475940 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3c410c4-0152-44b8-995c-75bb6e1d6f16-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb\" (UID: \"a3c410c4-0152-44b8-995c-75bb6e1d6f16\") " pod="openshift-ingress/payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb" Aug 04 11:12:34.576596 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.576565 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47026a7c-8e2c-4c7e-ac69-4e027cc868aa-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m\" (UID: \"47026a7c-8e2c-4c7e-ac69-4e027cc868aa\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m" Aug 04 11:12:34.576596 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.576597 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkwjt\" (UniqueName: \"kubernetes.io/projected/47026a7c-8e2c-4c7e-ac69-4e027cc868aa-kube-api-access-xkwjt\") pod \"payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m\" (UID: \"47026a7c-8e2c-4c7e-ac69-4e027cc868aa\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m" Aug 04 11:12:34.576861 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.576655 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3c410c4-0152-44b8-995c-75bb6e1d6f16-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb\" (UID: \"a3c410c4-0152-44b8-995c-75bb6e1d6f16\") " pod="openshift-ingress/payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb" Aug 04 11:12:34.576861 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.576770 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l7kg\" (UniqueName: \"kubernetes.io/projected/a3c410c4-0152-44b8-995c-75bb6e1d6f16-kube-api-access-8l7kg\") pod \"payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb\" (UID: \"a3c410c4-0152-44b8-995c-75bb6e1d6f16\") " pod="openshift-ingress/payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb" Aug 04 11:12:34.577360 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.577342 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3c410c4-0152-44b8-995c-75bb6e1d6f16-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb\" (UID: \"a3c410c4-0152-44b8-995c-75bb6e1d6f16\") " pod="openshift-ingress/payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb" Aug 04 11:12:34.577360 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.577354 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47026a7c-8e2c-4c7e-ac69-4e027cc868aa-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m\" (UID: \"47026a7c-8e2c-4c7e-ac69-4e027cc868aa\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m" Aug 04 11:12:34.584767 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.584739 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkwjt\" (UniqueName: \"kubernetes.io/projected/47026a7c-8e2c-4c7e-ac69-4e027cc868aa-kube-api-access-xkwjt\") pod \"payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m\" (UID: \"47026a7c-8e2c-4c7e-ac69-4e027cc868aa\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m" Aug 04 11:12:34.584872 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.584782 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l7kg\" (UniqueName: \"kubernetes.io/projected/a3c410c4-0152-44b8-995c-75bb6e1d6f16-kube-api-access-8l7kg\") pod \"payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb\" (UID: \"a3c410c4-0152-44b8-995c-75bb6e1d6f16\") " pod="openshift-ingress/payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb" Aug 04 11:12:34.619481 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.619447 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m" Aug 04 11:12:34.662251 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.662218 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb" Aug 04 11:12:34.761736 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.761703 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m"] Aug 04 11:12:34.764704 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:12:34.764674 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47026a7c_8e2c_4c7e_ac69_4e027cc868aa.slice/crio-41e61a48f503ef89b53e8b12e59b2045776cdbbf983164905df04db514e9294c WatchSource:0}: Error finding container 41e61a48f503ef89b53e8b12e59b2045776cdbbf983164905df04db514e9294c: Status 404 returned error can't find the container with id 41e61a48f503ef89b53e8b12e59b2045776cdbbf983164905df04db514e9294c Aug 04 11:12:34.816281 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:34.816260 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb"] Aug 04 11:12:34.819661 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:12:34.819627 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3c410c4_0152_44b8_995c_75bb6e1d6f16.slice/crio-89098fde57c324a0ff2cb6c853959b2cc18d9cdbe5240c5f6f6d4483aa2c4651 WatchSource:0}: Error finding container 89098fde57c324a0ff2cb6c853959b2cc18d9cdbe5240c5f6f6d4483aa2c4651: Status 404 returned error can't find the container with id 89098fde57c324a0ff2cb6c853959b2cc18d9cdbe5240c5f6f6d4483aa2c4651 Aug 04 11:12:35.619792 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:35.619759 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb" event={"ID":"a3c410c4-0152-44b8-995c-75bb6e1d6f16","Type":"ContainerStarted","Data":"1f6fdd1450ce6fc3a73f713d535128434172024d6aa5b1c22e9ef9ace2aaa8c3"} Aug 04 11:12:35.619979 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:35.619809 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb" event={"ID":"a3c410c4-0152-44b8-995c-75bb6e1d6f16","Type":"ContainerStarted","Data":"89098fde57c324a0ff2cb6c853959b2cc18d9cdbe5240c5f6f6d4483aa2c4651"} Aug 04 11:12:35.619979 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:35.619865 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb" Aug 04 11:12:35.621301 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:35.621269 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m" event={"ID":"47026a7c-8e2c-4c7e-ac69-4e027cc868aa","Type":"ContainerStarted","Data":"9d2ae7b661a9340c1dd86b18118b32aba1ef789b1b60d47ef33d4b29046a5c36"} Aug 04 11:12:35.621419 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:35.621313 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m" event={"ID":"47026a7c-8e2c-4c7e-ac69-4e027cc868aa","Type":"ContainerStarted","Data":"41e61a48f503ef89b53e8b12e59b2045776cdbbf983164905df04db514e9294c"} Aug 04 11:12:35.621419 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:35.621342 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m" Aug 04 11:12:35.635821 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:35.635776 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb" podStartSLOduration=1.6357575199999999 podStartE2EDuration="1.63575752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:12:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:12:35.634599168 +0000 UTC m=+2693.608681715" watchObservedRunningTime="2026-08-04 11:12:35.63575752 +0000 UTC m=+2693.609840048" Aug 04 11:12:35.649088 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:35.649048 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m" podStartSLOduration=1.6490360499999999 podStartE2EDuration="1.64903605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:12:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:12:35.647770378 +0000 UTC m=+2693.621852904" watchObservedRunningTime="2026-08-04 11:12:35.64903605 +0000 UTC m=+2693.623118577" Aug 04 11:12:46.627349 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:46.627319 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb" Aug 04 11:12:46.627825 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:46.627402 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m" Aug 04 11:12:57.466940 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:57.466906 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:12:57.808992 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:57.808957 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:12:58.143697 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:58.143610 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:12:58.249909 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:12:58.249878 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:13:00.615556 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:00.615520 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:13:00.714645 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:00.714617 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:13:01.738039 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:01.738005 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:13:02.083396 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:02.083355 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:13:02.293772 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:02.293735 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:13:02.414841 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:02.414756 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:13:02.462475 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:02.462442 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f"] Aug 04 11:13:02.466058 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:02.466040 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f" Aug 04 11:13:02.468077 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:02.468059 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-460cac\"" Aug 04 11:13:02.468195 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:02.468180 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-460cac-dockercfg-2bznq\"" Aug 04 11:13:02.482866 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:02.482837 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f"] Aug 04 11:13:02.513152 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:02.513124 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da977de0-cee3-4fbb-b5ca-94c23f5884dc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f\" (UID: \"da977de0-cee3-4fbb-b5ca-94c23f5884dc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f" Aug 04 11:13:02.513152 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:02.513153 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvnq9\" (UniqueName: \"kubernetes.io/projected/da977de0-cee3-4fbb-b5ca-94c23f5884dc-kube-api-access-xvnq9\") pod \"payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f\" (UID: \"da977de0-cee3-4fbb-b5ca-94c23f5884dc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f" Aug 04 11:13:02.550050 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:02.550014 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs"] Aug 04 11:13:02.553423 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:02.553405 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs" Aug 04 11:13:02.563189 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:02.563164 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs"] Aug 04 11:13:02.614206 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:02.614171 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da977de0-cee3-4fbb-b5ca-94c23f5884dc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f\" (UID: \"da977de0-cee3-4fbb-b5ca-94c23f5884dc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f" Aug 04 11:13:02.614206 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:02.614209 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvnq9\" (UniqueName: \"kubernetes.io/projected/da977de0-cee3-4fbb-b5ca-94c23f5884dc-kube-api-access-xvnq9\") pod \"payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f\" (UID: \"da977de0-cee3-4fbb-b5ca-94c23f5884dc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f" Aug 04 11:13:02.614438 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:02.614241 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80d7c11a-a33a-473e-992f-da9d3f3f982a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs\" (UID: \"80d7c11a-a33a-473e-992f-da9d3f3f982a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs" Aug 04 11:13:02.614438 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:02.614280 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2jk4\" (UniqueName: \"kubernetes.io/projected/80d7c11a-a33a-473e-992f-da9d3f3f982a-kube-api-access-h2jk4\") pod \"payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs\" (UID: \"80d7c11a-a33a-473e-992f-da9d3f3f982a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs" Aug 04 11:13:02.614965 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:02.614945 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da977de0-cee3-4fbb-b5ca-94c23f5884dc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f\" (UID: \"da977de0-cee3-4fbb-b5ca-94c23f5884dc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f" Aug 04 11:13:02.622625 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:02.622600 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvnq9\" (UniqueName: \"kubernetes.io/projected/da977de0-cee3-4fbb-b5ca-94c23f5884dc-kube-api-access-xvnq9\") pod \"payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f\" (UID: \"da977de0-cee3-4fbb-b5ca-94c23f5884dc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f" Aug 04 11:13:02.714817 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:02.714729 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80d7c11a-a33a-473e-992f-da9d3f3f982a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs\" (UID: \"80d7c11a-a33a-473e-992f-da9d3f3f982a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs" Aug 04 11:13:02.714817 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:02.714784 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2jk4\" (UniqueName: \"kubernetes.io/projected/80d7c11a-a33a-473e-992f-da9d3f3f982a-kube-api-access-h2jk4\") pod \"payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs\" (UID: \"80d7c11a-a33a-473e-992f-da9d3f3f982a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs" Aug 04 11:13:02.715614 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:02.715584 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80d7c11a-a33a-473e-992f-da9d3f3f982a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs\" (UID: \"80d7c11a-a33a-473e-992f-da9d3f3f982a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs" Aug 04 11:13:02.722512 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:02.722473 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2jk4\" (UniqueName: \"kubernetes.io/projected/80d7c11a-a33a-473e-992f-da9d3f3f982a-kube-api-access-h2jk4\") pod \"payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs\" (UID: \"80d7c11a-a33a-473e-992f-da9d3f3f982a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs" Aug 04 11:13:02.776334 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:02.776307 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f" Aug 04 11:13:02.865852 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:02.865823 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs" Aug 04 11:13:02.922768 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:02.922239 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f"] Aug 04 11:13:02.922911 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:13:02.922838 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda977de0_cee3_4fbb_b5ca_94c23f5884dc.slice/crio-514306754cfc22437d17576bca49251f20efbfc17c351a74f4336b894771cf4d WatchSource:0}: Error finding container 514306754cfc22437d17576bca49251f20efbfc17c351a74f4336b894771cf4d: Status 404 returned error can't find the container with id 514306754cfc22437d17576bca49251f20efbfc17c351a74f4336b894771cf4d Aug 04 11:13:03.025902 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:03.025880 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs"] Aug 04 11:13:03.027993 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:13:03.027963 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80d7c11a_a33a_473e_992f_da9d3f3f982a.slice/crio-1c13e1a45c7c934bc6bdd0ea493a967ac812806e31c014f1960837191f820b6b WatchSource:0}: Error finding container 1c13e1a45c7c934bc6bdd0ea493a967ac812806e31c014f1960837191f820b6b: Status 404 returned error can't find the container with id 1c13e1a45c7c934bc6bdd0ea493a967ac812806e31c014f1960837191f820b6b Aug 04 11:13:03.765238 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:03.765199 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs" event={"ID":"80d7c11a-a33a-473e-992f-da9d3f3f982a","Type":"ContainerStarted","Data":"1be1834aaff81809128f68f87fa625128df368105ea5eceebcdf0672ca82bc0e"} Aug 04 11:13:03.765444 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:03.765259 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs" event={"ID":"80d7c11a-a33a-473e-992f-da9d3f3f982a","Type":"ContainerStarted","Data":"1c13e1a45c7c934bc6bdd0ea493a967ac812806e31c014f1960837191f820b6b"} Aug 04 11:13:03.765444 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:03.765322 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs" Aug 04 11:13:03.766922 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:03.766899 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f" event={"ID":"da977de0-cee3-4fbb-b5ca-94c23f5884dc","Type":"ContainerStarted","Data":"0aa3664d2394917501cd7f473e580c76b274c06d2749676de6ac1c9657ae4343"} Aug 04 11:13:03.767037 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:03.766935 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f" event={"ID":"da977de0-cee3-4fbb-b5ca-94c23f5884dc","Type":"ContainerStarted","Data":"514306754cfc22437d17576bca49251f20efbfc17c351a74f4336b894771cf4d"} Aug 04 11:13:03.767098 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:03.767041 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f" Aug 04 11:13:03.784178 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:03.784128 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs" podStartSLOduration=1.784115575 podStartE2EDuration="1.784115575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:13:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:13:03.781603834 +0000 UTC m=+2721.755686361" watchObservedRunningTime="2026-08-04 11:13:03.784115575 +0000 UTC m=+2721.758198101" Aug 04 11:13:03.799768 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:03.799722 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f" podStartSLOduration=1.799708956 podStartE2EDuration="1.799708956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:13:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:13:03.797776242 +0000 UTC m=+2721.771858790" watchObservedRunningTime="2026-08-04 11:13:03.799708956 +0000 UTC m=+2721.773791483" Aug 04 11:13:14.773677 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:14.773639 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f" Aug 04 11:13:14.774112 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:14.773716 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs" Aug 04 11:13:25.703905 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:25.703861 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:13:35.173569 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:35.173527 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:13:53.157320 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:13:53.157280 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:14:23.897926 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:14:23.897880 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:14:23.998753 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:14:23.998723 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:14:43.140059 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:14:43.140008 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:14:43.208760 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:14:43.208725 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:15:00.137438 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:00.137403 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764035-lc5g4"] Aug 04 11:15:00.140754 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:00.140729 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-460cac-29764035-fmbls"] Aug 04 11:15:00.140894 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:00.140872 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764035-lc5g4" Aug 04 11:15:00.142810 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:00.142784 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qptbt\"" Aug 04 11:15:00.143789 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:00.143772 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bc4a78-29764035-g5dm2"] Aug 04 11:15:00.143900 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:00.143877 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-460cac-29764035-fmbls" Aug 04 11:15:00.155221 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:00.155200 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764035-lc5g4"] Aug 04 11:15:00.155404 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:00.155390 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bc4a78-29764035-g5dm2"] Aug 04 11:15:00.155543 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:00.155402 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bc4a78-29764035-g5dm2" Aug 04 11:15:00.158185 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:00.158157 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-460cac-29764035-fmbls"] Aug 04 11:15:00.159573 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:00.159551 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764035-lc5g4" Aug 04 11:15:00.171336 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:00.171320 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-460cac-29764035-fmbls" Aug 04 11:15:00.208188 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:00.208162 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bc4a78-29764035-g5dm2" Aug 04 11:15:00.321471 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:00.321437 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29764035-lc5g4"] Aug 04 11:15:00.324029 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:15:00.323993 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcac02ebe_61ae_4524_8ae1_4473d1596545.slice/crio-d0824b45322e526ac3db58fba498395bbcea782c2920b036483c39b7aa0f6088 WatchSource:0}: Error finding container d0824b45322e526ac3db58fba498395bbcea782c2920b036483c39b7aa0f6088: Status 404 returned error can't find the container with id d0824b45322e526ac3db58fba498395bbcea782c2920b036483c39b7aa0f6088 Aug 04 11:15:00.380574 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:00.380545 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bc4a78-29764035-g5dm2"] Aug 04 11:15:00.382794 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:15:00.382762 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4f86870_bc42_419a_902b_dba73521e0e9.slice/crio-b7272dbf85d5e36142e100b45f9220fe18cb9c2ec200f26cea0a312da0fd1eef WatchSource:0}: Error finding container b7272dbf85d5e36142e100b45f9220fe18cb9c2ec200f26cea0a312da0fd1eef: Status 404 returned error can't find the container with id b7272dbf85d5e36142e100b45f9220fe18cb9c2ec200f26cea0a312da0fd1eef Aug 04 11:15:00.559125 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:00.559092 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-460cac-29764035-fmbls"] Aug 04 11:15:00.561127 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:15:00.561103 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd88a2c93_a89f_4001_8f90_7fb8ef86bf60.slice/crio-9c51bae87e6de0619a2e870da85ccfb737098e7009ce19207553e89295f2d0ae WatchSource:0}: Error finding container 9c51bae87e6de0619a2e870da85ccfb737098e7009ce19207553e89295f2d0ae: Status 404 returned error can't find the container with id 9c51bae87e6de0619a2e870da85ccfb737098e7009ce19207553e89295f2d0ae Aug 04 11:15:01.315804 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:01.315771 2535 generic.go:356] "Generic (PLEG): container finished" podID="b4f86870-bc42-419a-902b-dba73521e0e9" containerID="cf8a7694901996a3664b41fef6a48e126d70d3089179c8be3c7e94fd55d65155" exitCode=0 Aug 04 11:15:01.316262 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:01.315842 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bc4a78-29764035-g5dm2" event={"ID":"b4f86870-bc42-419a-902b-dba73521e0e9","Type":"ContainerDied","Data":"cf8a7694901996a3664b41fef6a48e126d70d3089179c8be3c7e94fd55d65155"} Aug 04 11:15:01.316262 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:01.315886 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bc4a78-29764035-g5dm2" event={"ID":"b4f86870-bc42-419a-902b-dba73521e0e9","Type":"ContainerStarted","Data":"b7272dbf85d5e36142e100b45f9220fe18cb9c2ec200f26cea0a312da0fd1eef"} Aug 04 11:15:01.317311 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:01.317284 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-460cac-29764035-fmbls" event={"ID":"d88a2c93-a89f-4001-8f90-7fb8ef86bf60","Type":"ContainerStarted","Data":"374e690473ac8b4ba351bc5ae08ca18c1d6c7352917b812ae605e8ab5f18030a"} Aug 04 11:15:01.317402 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:01.317335 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-460cac-29764035-fmbls" event={"ID":"d88a2c93-a89f-4001-8f90-7fb8ef86bf60","Type":"ContainerStarted","Data":"9c51bae87e6de0619a2e870da85ccfb737098e7009ce19207553e89295f2d0ae"} Aug 04 11:15:01.318890 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:01.318867 2535 generic.go:356] "Generic (PLEG): container finished" podID="cac02ebe-61ae-4524-8ae1-4473d1596545" containerID="eaad773e9ae142d5fd9bed1b44e9209b435f3bc03f686dbf443f626f10c7480a" exitCode=0 Aug 04 11:15:01.318971 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:01.318928 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764035-lc5g4" event={"ID":"cac02ebe-61ae-4524-8ae1-4473d1596545","Type":"ContainerDied","Data":"eaad773e9ae142d5fd9bed1b44e9209b435f3bc03f686dbf443f626f10c7480a"} Aug 04 11:15:01.318971 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:01.318957 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764035-lc5g4" event={"ID":"cac02ebe-61ae-4524-8ae1-4473d1596545","Type":"ContainerStarted","Data":"d0824b45322e526ac3db58fba498395bbcea782c2920b036483c39b7aa0f6088"} Aug 04 11:15:01.357250 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:01.357194 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-460cac-29764035-fmbls" podStartSLOduration=1.3571788790000001 podStartE2EDuration="1.357178879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:15:01.354925381 +0000 UTC m=+2839.329007932" watchObservedRunningTime="2026-08-04 11:15:01.357178879 +0000 UTC m=+2839.331261406" Aug 04 11:15:02.324877 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:02.324844 2535 generic.go:356] "Generic (PLEG): container finished" podID="d88a2c93-a89f-4001-8f90-7fb8ef86bf60" containerID="374e690473ac8b4ba351bc5ae08ca18c1d6c7352917b812ae605e8ab5f18030a" exitCode=0 Aug 04 11:15:02.325285 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:02.324935 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-460cac-29764035-fmbls" event={"ID":"d88a2c93-a89f-4001-8f90-7fb8ef86bf60","Type":"ContainerDied","Data":"374e690473ac8b4ba351bc5ae08ca18c1d6c7352917b812ae605e8ab5f18030a"} Aug 04 11:15:02.473215 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:02.473195 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bc4a78-29764035-g5dm2" Aug 04 11:15:02.496839 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:02.496811 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764035-lc5g4" Aug 04 11:15:03.336552 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:03.336516 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bc4a78-29764035-g5dm2" Aug 04 11:15:03.336552 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:03.336533 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bc4a78-29764035-g5dm2" event={"ID":"b4f86870-bc42-419a-902b-dba73521e0e9","Type":"ContainerDied","Data":"b7272dbf85d5e36142e100b45f9220fe18cb9c2ec200f26cea0a312da0fd1eef"} Aug 04 11:15:03.337029 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:03.336610 2535 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b7272dbf85d5e36142e100b45f9220fe18cb9c2ec200f26cea0a312da0fd1eef" Aug 04 11:15:03.337958 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:03.337930 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764035-lc5g4" Aug 04 11:15:03.338073 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:03.337984 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29764035-lc5g4" event={"ID":"cac02ebe-61ae-4524-8ae1-4473d1596545","Type":"ContainerDied","Data":"d0824b45322e526ac3db58fba498395bbcea782c2920b036483c39b7aa0f6088"} Aug 04 11:15:03.338073 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:03.338032 2535 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0824b45322e526ac3db58fba498395bbcea782c2920b036483c39b7aa0f6088" Aug 04 11:15:03.476748 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:03.476725 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-460cac-29764035-fmbls" Aug 04 11:15:04.344371 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:04.344341 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-460cac-29764035-fmbls" Aug 04 11:15:04.344371 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:04.344345 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-460cac-29764035-fmbls" event={"ID":"d88a2c93-a89f-4001-8f90-7fb8ef86bf60","Type":"ContainerDied","Data":"9c51bae87e6de0619a2e870da85ccfb737098e7009ce19207553e89295f2d0ae"} Aug 04 11:15:04.344859 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:15:04.344400 2535 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9c51bae87e6de0619a2e870da85ccfb737098e7009ce19207553e89295f2d0ae" Aug 04 11:16:58.755514 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:16:58.755455 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:17:00.489678 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:17:00.489637 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:17:19.049405 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:17:19.049372 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:17:42.303477 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:17:42.303407 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:17:56.165298 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:17:56.165260 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:17:59.135658 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:17:59.135621 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:18:41.611448 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:18:41.611416 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:18:52.015052 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:18:52.015012 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:18:53.570258 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:18:53.570204 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:19:04.869123 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:19:04.869070 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:19:25.983301 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:19:25.983250 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:20:02.105172 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:20:02.105120 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:20:45.639191 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:20:45.639157 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:20:46.581892 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:20:46.581862 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:21:05.946084 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:05.946042 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:21:06.275745 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:06.275707 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:21:06.916609 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:06.916572 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:21:09.082253 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:09.082216 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:21:09.171842 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:09.171806 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:21:10.191133 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:10.191096 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:21:10.276709 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:10.276678 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:21:10.732866 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:10.732831 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:21:10.815869 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:10.815836 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:21:10.816943 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:10.816922 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7"] Aug 04 11:21:10.817342 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:10.817330 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d88a2c93-a89f-4001-8f90-7fb8ef86bf60" containerName="cleanup" Aug 04 11:21:10.817383 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:10.817344 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="d88a2c93-a89f-4001-8f90-7fb8ef86bf60" containerName="cleanup" Aug 04 11:21:10.817383 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:10.817357 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cac02ebe-61ae-4524-8ae1-4473d1596545" containerName="cleanup" Aug 04 11:21:10.817383 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:10.817362 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="cac02ebe-61ae-4524-8ae1-4473d1596545" containerName="cleanup" Aug 04 11:21:10.817383 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:10.817368 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4f86870-bc42-419a-902b-dba73521e0e9" containerName="cleanup" Aug 04 11:21:10.817383 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:10.817375 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4f86870-bc42-419a-902b-dba73521e0e9" containerName="cleanup" Aug 04 11:21:10.817572 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:10.817452 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4f86870-bc42-419a-902b-dba73521e0e9" containerName="cleanup" Aug 04 11:21:10.817572 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:10.817459 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="d88a2c93-a89f-4001-8f90-7fb8ef86bf60" containerName="cleanup" Aug 04 11:21:10.817572 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:10.817469 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="cac02ebe-61ae-4524-8ae1-4473d1596545" containerName="cleanup" Aug 04 11:21:10.820777 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:10.820758 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7" Aug 04 11:21:10.823207 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:10.823185 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-1cb57d\"" Aug 04 11:21:10.834961 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:10.834934 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7"] Aug 04 11:21:10.836807 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:10.836786 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw"] Aug 04 11:21:10.840057 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:10.840040 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw" Aug 04 11:21:10.842091 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:10.842074 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-1cb57d-dockercfg-mgg7g\"" Aug 04 11:21:10.849519 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:10.849500 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw"] Aug 04 11:21:10.956737 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:10.956700 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj8wg\" (UniqueName: \"kubernetes.io/projected/ca6c79c0-045c-4d38-a3eb-4a4c80adb97a-kube-api-access-nj8wg\") pod \"payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7\" (UID: \"ca6c79c0-045c-4d38-a3eb-4a4c80adb97a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7" Aug 04 11:21:10.956930 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:10.956765 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcxd4\" (UniqueName: \"kubernetes.io/projected/9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a-kube-api-access-tcxd4\") pod \"payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw\" (UID: \"9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw" Aug 04 11:21:10.956930 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:10.956846 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca6c79c0-045c-4d38-a3eb-4a4c80adb97a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7\" (UID: \"ca6c79c0-045c-4d38-a3eb-4a4c80adb97a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7" Aug 04 11:21:10.957027 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:10.956939 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw\" (UID: \"9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw" Aug 04 11:21:11.057773 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:11.057736 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca6c79c0-045c-4d38-a3eb-4a4c80adb97a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7\" (UID: \"ca6c79c0-045c-4d38-a3eb-4a4c80adb97a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7" Aug 04 11:21:11.057949 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:11.057799 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw\" (UID: \"9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw" Aug 04 11:21:11.057949 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:11.057836 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj8wg\" (UniqueName: \"kubernetes.io/projected/ca6c79c0-045c-4d38-a3eb-4a4c80adb97a-kube-api-access-nj8wg\") pod \"payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7\" (UID: \"ca6c79c0-045c-4d38-a3eb-4a4c80adb97a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7" Aug 04 11:21:11.057949 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:11.057861 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcxd4\" (UniqueName: \"kubernetes.io/projected/9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a-kube-api-access-tcxd4\") pod \"payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw\" (UID: \"9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw" Aug 04 11:21:11.058443 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:11.058424 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca6c79c0-045c-4d38-a3eb-4a4c80adb97a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7\" (UID: \"ca6c79c0-045c-4d38-a3eb-4a4c80adb97a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7" Aug 04 11:21:11.058516 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:11.058466 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw\" (UID: \"9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw" Aug 04 11:21:11.065730 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:11.065704 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcxd4\" (UniqueName: \"kubernetes.io/projected/9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a-kube-api-access-tcxd4\") pod \"payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw\" (UID: \"9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw" Aug 04 11:21:11.065833 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:11.065795 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj8wg\" (UniqueName: \"kubernetes.io/projected/ca6c79c0-045c-4d38-a3eb-4a4c80adb97a-kube-api-access-nj8wg\") pod \"payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7\" (UID: \"ca6c79c0-045c-4d38-a3eb-4a4c80adb97a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7" Aug 04 11:21:11.132081 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:11.132049 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7" Aug 04 11:21:11.153207 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:11.153185 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw" Aug 04 11:21:11.314600 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:11.314577 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw"] Aug 04 11:21:11.316646 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:21:11.316622 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cf3792d_8f3e_4bf1_8a23_161ed41b8e1a.slice/crio-2a1f5c4ccee86162059e2a57675dd6b51bcd4a888ad999aac9a1c5bd83790ff1 WatchSource:0}: Error finding container 2a1f5c4ccee86162059e2a57675dd6b51bcd4a888ad999aac9a1c5bd83790ff1: Status 404 returned error can't find the container with id 2a1f5c4ccee86162059e2a57675dd6b51bcd4a888ad999aac9a1c5bd83790ff1 Aug 04 11:21:11.488914 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:11.488885 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7"] Aug 04 11:21:11.491202 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:21:11.491170 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca6c79c0_045c_4d38_a3eb_4a4c80adb97a.slice/crio-c2c4a3e88d8f911c2743ddebf97b57b5d4bc7d051062f25472809dab5b3bca6e WatchSource:0}: Error finding container c2c4a3e88d8f911c2743ddebf97b57b5d4bc7d051062f25472809dab5b3bca6e: Status 404 returned error can't find the container with id c2c4a3e88d8f911c2743ddebf97b57b5d4bc7d051062f25472809dab5b3bca6e Aug 04 11:21:12.121673 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:12.121637 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw" event={"ID":"9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a","Type":"ContainerStarted","Data":"deb9ec9cb649c68bf7d2365708c8fd1d488f90a0b41b4db6aa026e366a1519b5"} Aug 04 11:21:12.121868 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:12.121703 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw" event={"ID":"9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a","Type":"ContainerStarted","Data":"2a1f5c4ccee86162059e2a57675dd6b51bcd4a888ad999aac9a1c5bd83790ff1"} Aug 04 11:21:12.121868 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:12.121747 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw" Aug 04 11:21:12.123255 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:12.123232 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7" event={"ID":"ca6c79c0-045c-4d38-a3eb-4a4c80adb97a","Type":"ContainerStarted","Data":"0638a431768d8753daeef11ca7d5869c3dc7d93eb8bd59936602a489ed79c58f"} Aug 04 11:21:12.123381 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:12.123265 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7" event={"ID":"ca6c79c0-045c-4d38-a3eb-4a4c80adb97a","Type":"ContainerStarted","Data":"c2c4a3e88d8f911c2743ddebf97b57b5d4bc7d051062f25472809dab5b3bca6e"} Aug 04 11:21:12.123381 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:12.123299 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7" Aug 04 11:21:12.141224 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:12.141179 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw" podStartSLOduration=2.141166154 podStartE2EDuration="2.141166154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:21:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:21:12.138785629 +0000 UTC m=+3210.112868180" watchObservedRunningTime="2026-08-04 11:21:12.141166154 +0000 UTC m=+3210.115248682" Aug 04 11:21:12.156994 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:12.156941 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7" podStartSLOduration=2.156924188 podStartE2EDuration="2.156924188s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:21:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:21:12.154724848 +0000 UTC m=+3210.128807377" watchObservedRunningTime="2026-08-04 11:21:12.156924188 +0000 UTC m=+3210.131006722" Aug 04 11:21:23.129656 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:23.129619 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7" Aug 04 11:21:23.130230 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:23.129688 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw" Aug 04 11:21:34.094599 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:34.094563 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:21:34.771110 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:34.771079 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:21:35.097180 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:35.097152 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:21:35.160435 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:35.160405 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:21:38.184420 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:38.184377 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:21:38.272595 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:38.272566 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:21:39.308626 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:39.308590 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:21:39.814961 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:39.814928 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:21:39.912139 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:39.912096 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:21:39.950062 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:39.950024 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s"] Aug 04 11:21:39.953428 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:39.953407 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s" Aug 04 11:21:39.955675 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:39.955656 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-651a7a\"" Aug 04 11:21:39.955809 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:39.955748 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-651a7a-dockercfg-2m449\"" Aug 04 11:21:39.965035 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:39.965012 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s"] Aug 04 11:21:39.987864 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:39.987829 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5"] Aug 04 11:21:39.992630 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:39.992596 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:21:39.994061 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:39.994035 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5" Aug 04 11:21:40.004044 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:40.004023 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5"] Aug 04 11:21:40.016336 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:40.016305 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/263ebf50-1233-463f-9c5b-d021021ad60d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s\" (UID: \"263ebf50-1233-463f-9c5b-d021021ad60d\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s" Aug 04 11:21:40.016458 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:40.016399 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qkg5\" (UniqueName: \"kubernetes.io/projected/263ebf50-1233-463f-9c5b-d021021ad60d-kube-api-access-5qkg5\") pod \"payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s\" (UID: \"263ebf50-1233-463f-9c5b-d021021ad60d\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s" Aug 04 11:21:40.117727 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:40.117638 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/263ebf50-1233-463f-9c5b-d021021ad60d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s\" (UID: \"263ebf50-1233-463f-9c5b-d021021ad60d\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s" Aug 04 11:21:40.117727 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:40.117695 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ca9eaa7-e647-46c8-8474-748a69e4a737-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5\" (UID: \"1ca9eaa7-e647-46c8-8474-748a69e4a737\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5" Aug 04 11:21:40.117727 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:40.117721 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qkg5\" (UniqueName: \"kubernetes.io/projected/263ebf50-1233-463f-9c5b-d021021ad60d-kube-api-access-5qkg5\") pod \"payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s\" (UID: \"263ebf50-1233-463f-9c5b-d021021ad60d\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s" Aug 04 11:21:40.117947 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:40.117766 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82hm9\" (UniqueName: \"kubernetes.io/projected/1ca9eaa7-e647-46c8-8474-748a69e4a737-kube-api-access-82hm9\") pod \"payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5\" (UID: \"1ca9eaa7-e647-46c8-8474-748a69e4a737\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5" Aug 04 11:21:40.118397 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:40.118377 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/263ebf50-1233-463f-9c5b-d021021ad60d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s\" (UID: \"263ebf50-1233-463f-9c5b-d021021ad60d\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s" Aug 04 11:21:40.125682 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:40.125656 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qkg5\" (UniqueName: \"kubernetes.io/projected/263ebf50-1233-463f-9c5b-d021021ad60d-kube-api-access-5qkg5\") pod \"payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s\" (UID: \"263ebf50-1233-463f-9c5b-d021021ad60d\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s" Aug 04 11:21:40.218287 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:40.218254 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82hm9\" (UniqueName: \"kubernetes.io/projected/1ca9eaa7-e647-46c8-8474-748a69e4a737-kube-api-access-82hm9\") pod \"payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5\" (UID: \"1ca9eaa7-e647-46c8-8474-748a69e4a737\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5" Aug 04 11:21:40.218512 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:40.218337 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ca9eaa7-e647-46c8-8474-748a69e4a737-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5\" (UID: \"1ca9eaa7-e647-46c8-8474-748a69e4a737\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5" Aug 04 11:21:40.219036 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:40.219017 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ca9eaa7-e647-46c8-8474-748a69e4a737-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5\" (UID: \"1ca9eaa7-e647-46c8-8474-748a69e4a737\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5" Aug 04 11:21:40.225952 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:40.225931 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82hm9\" (UniqueName: \"kubernetes.io/projected/1ca9eaa7-e647-46c8-8474-748a69e4a737-kube-api-access-82hm9\") pod \"payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5\" (UID: \"1ca9eaa7-e647-46c8-8474-748a69e4a737\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5" Aug 04 11:21:40.264738 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:40.264709 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s" Aug 04 11:21:40.307437 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:40.307401 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5" Aug 04 11:21:40.411247 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:40.411216 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s"] Aug 04 11:21:40.466209 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:40.466160 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5"] Aug 04 11:21:40.468800 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:21:40.468772 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ca9eaa7_e647_46c8_8474_748a69e4a737.slice/crio-08fd7635f0f659085bfc35ba899643862a1e6c56a6b98ac149b18e82fc2e9708 WatchSource:0}: Error finding container 08fd7635f0f659085bfc35ba899643862a1e6c56a6b98ac149b18e82fc2e9708: Status 404 returned error can't find the container with id 08fd7635f0f659085bfc35ba899643862a1e6c56a6b98ac149b18e82fc2e9708 Aug 04 11:21:41.276655 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:41.276620 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s" event={"ID":"263ebf50-1233-463f-9c5b-d021021ad60d","Type":"ContainerStarted","Data":"f814380822f158215b424ee674c7210f9c9c2bc7c53c3ad6987b1b88cfc28572"} Aug 04 11:21:41.276869 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:41.276792 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s" Aug 04 11:21:41.276869 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:41.276826 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s" event={"ID":"263ebf50-1233-463f-9c5b-d021021ad60d","Type":"ContainerStarted","Data":"e7be6ddb12751b1aa4c019a40feb77c7fc3f1b6d9e7b81bb88334a8b2943e450"} Aug 04 11:21:41.278234 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:41.278208 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5" event={"ID":"1ca9eaa7-e647-46c8-8474-748a69e4a737","Type":"ContainerStarted","Data":"8b0a023a9c5b66e0b267e3037de76461c0b1aa326b5225fb1c2d248c383a2dbe"} Aug 04 11:21:41.278360 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:41.278249 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5" Aug 04 11:21:41.278360 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:41.278271 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5" event={"ID":"1ca9eaa7-e647-46c8-8474-748a69e4a737","Type":"ContainerStarted","Data":"08fd7635f0f659085bfc35ba899643862a1e6c56a6b98ac149b18e82fc2e9708"} Aug 04 11:21:41.293410 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:41.293359 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s" podStartSLOduration=2.293345208 podStartE2EDuration="2.293345208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:21:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:21:41.292031664 +0000 UTC m=+3239.266114217" watchObservedRunningTime="2026-08-04 11:21:41.293345208 +0000 UTC m=+3239.267427734" Aug 04 11:21:41.308326 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:41.308283 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5" podStartSLOduration=2.308266568 podStartE2EDuration="2.308266568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:21:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:21:41.306557731 +0000 UTC m=+3239.280640285" watchObservedRunningTime="2026-08-04 11:21:41.308266568 +0000 UTC m=+3239.282349140" Aug 04 11:21:52.284949 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:52.284917 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s" Aug 04 11:21:52.285421 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:52.285140 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5" Aug 04 11:21:58.316272 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:58.316237 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:21:58.379860 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:21:58.379829 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:22:03.192802 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:22:03.192771 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:22:10.150726 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:22:10.150686 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-whd9j_e2d7c7cf-b757-4023-a277-38570756f493/payload-processing/0.log" Aug 04 11:22:10.278291 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:22:10.278262 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s_263ebf50-1233-463f-9c5b-d021021ad60d/payload-processing/0.log" Aug 04 11:22:17.565811 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:22:17.565775 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:22:55.068856 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:22:55.068827 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw_9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a/payload-processing/0.log" Aug 04 11:22:55.197826 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:22:55.197797 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-whd9j_e2d7c7cf-b757-4023-a277-38570756f493/payload-processing/0.log" Aug 04 11:22:55.322174 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:22:55.322098 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s_263ebf50-1233-463f-9c5b-d021021ad60d/payload-processing/0.log" Aug 04 11:22:56.421140 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:22:56.421110 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:22:56.522077 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:22:56.522034 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:23:01.835025 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:01.834978 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:23:01.934459 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:01.934422 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:23:02.965456 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:02.965407 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:23:03.146646 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:03.146600 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:23:03.519227 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:03.519193 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:23:03.614607 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:03.614574 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:23:03.652150 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:03.652117 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw"] Aug 04 11:23:03.655764 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:03.655747 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw" Aug 04 11:23:03.658754 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:03.658732 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-a031b5-dockercfg-xvg2d\"" Aug 04 11:23:03.658874 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:03.658732 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-a031b5\"" Aug 04 11:23:03.665695 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:03.665672 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw"] Aug 04 11:23:03.670910 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:03.670888 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75v7z\" (UniqueName: \"kubernetes.io/projected/018e4ae9-9fd3-45e0-b211-2183ed1225c3-kube-api-access-75v7z\") pod \"payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw\" (UID: \"018e4ae9-9fd3-45e0-b211-2183ed1225c3\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw" Aug 04 11:23:03.671006 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:03.670955 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/018e4ae9-9fd3-45e0-b211-2183ed1225c3-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw\" (UID: \"018e4ae9-9fd3-45e0-b211-2183ed1225c3\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw" Aug 04 11:23:03.681156 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:03.681126 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq"] Aug 04 11:23:03.684629 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:03.684607 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq" Aug 04 11:23:03.696149 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:03.696117 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq"] Aug 04 11:23:03.771393 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:03.771300 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/018e4ae9-9fd3-45e0-b211-2183ed1225c3-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw\" (UID: \"018e4ae9-9fd3-45e0-b211-2183ed1225c3\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw" Aug 04 11:23:03.771393 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:03.771365 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvxr9\" (UniqueName: \"kubernetes.io/projected/0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e-kube-api-access-vvxr9\") pod \"payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq\" (UID: \"0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq" Aug 04 11:23:03.771644 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:03.771411 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq\" (UID: \"0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq" Aug 04 11:23:03.771644 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:03.771544 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75v7z\" (UniqueName: \"kubernetes.io/projected/018e4ae9-9fd3-45e0-b211-2183ed1225c3-kube-api-access-75v7z\") pod \"payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw\" (UID: \"018e4ae9-9fd3-45e0-b211-2183ed1225c3\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw" Aug 04 11:23:03.772081 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:03.772062 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/018e4ae9-9fd3-45e0-b211-2183ed1225c3-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw\" (UID: \"018e4ae9-9fd3-45e0-b211-2183ed1225c3\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw" Aug 04 11:23:03.779986 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:03.779957 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75v7z\" (UniqueName: \"kubernetes.io/projected/018e4ae9-9fd3-45e0-b211-2183ed1225c3-kube-api-access-75v7z\") pod \"payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw\" (UID: \"018e4ae9-9fd3-45e0-b211-2183ed1225c3\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw" Aug 04 11:23:03.872203 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:03.872167 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvxr9\" (UniqueName: \"kubernetes.io/projected/0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e-kube-api-access-vvxr9\") pod \"payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq\" (UID: \"0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq" Aug 04 11:23:03.872384 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:03.872221 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq\" (UID: \"0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq" Aug 04 11:23:03.873021 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:03.873000 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq\" (UID: \"0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq" Aug 04 11:23:03.880926 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:03.880898 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvxr9\" (UniqueName: \"kubernetes.io/projected/0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e-kube-api-access-vvxr9\") pod \"payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq\" (UID: \"0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq" Aug 04 11:23:03.968529 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:03.968467 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw" Aug 04 11:23:03.995828 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:03.995801 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq" Aug 04 11:23:04.339933 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:04.339900 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw"] Aug 04 11:23:04.340977 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:23:04.340933 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod018e4ae9_9fd3_45e0_b211_2183ed1225c3.slice/crio-fa4d79c7607d270d27c62ceb28977a544ef6072ee16d624f3e819cddcf281673 WatchSource:0}: Error finding container fa4d79c7607d270d27c62ceb28977a544ef6072ee16d624f3e819cddcf281673: Status 404 returned error can't find the container with id fa4d79c7607d270d27c62ceb28977a544ef6072ee16d624f3e819cddcf281673 Aug 04 11:23:04.368832 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:04.368777 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq"] Aug 04 11:23:04.371627 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:23:04.371601 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ef4c3fb_fd83_4789_be8b_bd5d4964ba6e.slice/crio-13d24d1e50650792890716f749742ef3b8367e47f13970dd66a65a9886f14a4b WatchSource:0}: Error finding container 13d24d1e50650792890716f749742ef3b8367e47f13970dd66a65a9886f14a4b: Status 404 returned error can't find the container with id 13d24d1e50650792890716f749742ef3b8367e47f13970dd66a65a9886f14a4b Aug 04 11:23:04.702176 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:04.702063 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw" event={"ID":"018e4ae9-9fd3-45e0-b211-2183ed1225c3","Type":"ContainerStarted","Data":"126f43a12bf1f7015cb2e8a73aa4e89f5d6c6ff99c1c6a57be1a9fe6c7bb1f7a"} Aug 04 11:23:04.702176 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:04.702130 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw" Aug 04 11:23:04.702176 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:04.702155 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw" event={"ID":"018e4ae9-9fd3-45e0-b211-2183ed1225c3","Type":"ContainerStarted","Data":"fa4d79c7607d270d27c62ceb28977a544ef6072ee16d624f3e819cddcf281673"} Aug 04 11:23:04.703655 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:04.703631 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq" event={"ID":"0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e","Type":"ContainerStarted","Data":"02500e6d072a5214f6047e30c4f402bf949ff36357f05d9204fa94ee03791f33"} Aug 04 11:23:04.703745 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:04.703668 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq" event={"ID":"0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e","Type":"ContainerStarted","Data":"13d24d1e50650792890716f749742ef3b8367e47f13970dd66a65a9886f14a4b"} Aug 04 11:23:04.703745 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:04.703692 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq" Aug 04 11:23:04.718115 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:04.718074 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw" podStartSLOduration=1.718060209 podStartE2EDuration="1.718060209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:23:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:23:04.716700345 +0000 UTC m=+3322.690782922" watchObservedRunningTime="2026-08-04 11:23:04.718060209 +0000 UTC m=+3322.692142739" Aug 04 11:23:04.730791 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:04.730738 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq" podStartSLOduration=1.730719402 podStartE2EDuration="1.730719402s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:23:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:23:04.73048223 +0000 UTC m=+3322.704564781" watchObservedRunningTime="2026-08-04 11:23:04.730719402 +0000 UTC m=+3322.704801929" Aug 04 11:23:15.711915 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:15.711881 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw" Aug 04 11:23:15.712289 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:15.712060 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq" Aug 04 11:23:26.867121 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:26.867088 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:23:27.580677 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:27.580642 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a031b5-t67gd"] Aug 04 11:23:27.584639 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:27.584618 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a031b5-t67gd" Aug 04 11:23:27.586519 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:27.586476 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qptbt\"" Aug 04 11:23:27.590923 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:27.590901 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a031b5-t67gd"] Aug 04 11:23:27.692022 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:27.691990 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50a45b17-d14b-4a69-be3b-6c73cd6d656f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-a031b5-t67gd\" (UID: \"50a45b17-d14b-4a69-be3b-6c73cd6d656f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a031b5-t67gd" Aug 04 11:23:27.792867 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:27.792830 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50a45b17-d14b-4a69-be3b-6c73cd6d656f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-a031b5-t67gd\" (UID: \"50a45b17-d14b-4a69-be3b-6c73cd6d656f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a031b5-t67gd" Aug 04 11:23:27.793478 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:27.793457 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50a45b17-d14b-4a69-be3b-6c73cd6d656f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-a031b5-t67gd\" (UID: \"50a45b17-d14b-4a69-be3b-6c73cd6d656f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a031b5-t67gd" Aug 04 11:23:27.897974 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:27.897893 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a031b5-t67gd" Aug 04 11:23:28.035041 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:28.035014 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a031b5-t67gd"] Aug 04 11:23:28.038209 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:23:28.038185 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50a45b17_d14b_4a69_be3b_6c73cd6d656f.slice/crio-dcc89b289160a2ddb208668082fc56a29c39ca0a623082a17c7fba68fd35be45 WatchSource:0}: Error finding container dcc89b289160a2ddb208668082fc56a29c39ca0a623082a17c7fba68fd35be45: Status 404 returned error can't find the container with id dcc89b289160a2ddb208668082fc56a29c39ca0a623082a17c7fba68fd35be45 Aug 04 11:23:28.827330 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:28.827302 2535 generic.go:356] "Generic (PLEG): container finished" podID="50a45b17-d14b-4a69-be3b-6c73cd6d656f" containerID="276f2f5ed0d72339cd88372a58018f249372b695aa8b8883b50b2d052103632f" exitCode=0 Aug 04 11:23:28.827456 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:28.827384 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a031b5-t67gd" event={"ID":"50a45b17-d14b-4a69-be3b-6c73cd6d656f","Type":"ContainerDied","Data":"276f2f5ed0d72339cd88372a58018f249372b695aa8b8883b50b2d052103632f"} Aug 04 11:23:28.827456 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:28.827433 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a031b5-t67gd" event={"ID":"50a45b17-d14b-4a69-be3b-6c73cd6d656f","Type":"ContainerStarted","Data":"dcc89b289160a2ddb208668082fc56a29c39ca0a623082a17c7fba68fd35be45"} Aug 04 11:23:29.976950 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:29.976928 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a031b5-t67gd" Aug 04 11:23:30.114854 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:30.114752 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50a45b17-d14b-4a69-be3b-6c73cd6d656f-maas-api-service-ca\") pod \"50a45b17-d14b-4a69-be3b-6c73cd6d656f\" (UID: \"50a45b17-d14b-4a69-be3b-6c73cd6d656f\") " Aug 04 11:23:30.115109 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:30.115088 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50a45b17-d14b-4a69-be3b-6c73cd6d656f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "50a45b17-d14b-4a69-be3b-6c73cd6d656f" (UID: "50a45b17-d14b-4a69-be3b-6c73cd6d656f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:23:30.215471 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:30.215441 2535 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50a45b17-d14b-4a69-be3b-6c73cd6d656f-maas-api-service-ca\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:23:30.838219 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:30.838182 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a031b5-t67gd" event={"ID":"50a45b17-d14b-4a69-be3b-6c73cd6d656f","Type":"ContainerDied","Data":"dcc89b289160a2ddb208668082fc56a29c39ca0a623082a17c7fba68fd35be45"} Aug 04 11:23:30.838219 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:30.838200 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a031b5-t67gd" Aug 04 11:23:30.838468 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:30.838230 2535 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dcc89b289160a2ddb208668082fc56a29c39ca0a623082a17c7fba68fd35be45" Aug 04 11:23:32.034023 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.033988 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:23:32.062533 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.062429 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw"] Aug 04 11:23:32.062700 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.062674 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw" podUID="018e4ae9-9fd3-45e0-b211-2183ed1225c3" containerName="payload-processing" containerID="cri-o://126f43a12bf1f7015cb2e8a73aa4e89f5d6c6ff99c1c6a57be1a9fe6c7bb1f7a" gracePeriod=30 Aug 04 11:23:32.069833 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.069800 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq"] Aug 04 11:23:32.070040 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.070012 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq" podUID="0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e" containerName="payload-pre-processing" containerID="cri-o://02500e6d072a5214f6047e30c4f402bf949ff36357f05d9204fa94ee03791f33" gracePeriod=30 Aug 04 11:23:32.099453 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.099420 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:23:32.171331 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.171295 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:23:32.215053 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.215009 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a031b5-t67gd"] Aug 04 11:23:32.220237 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.220209 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a031b5-t67gd"] Aug 04 11:23:32.270850 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.270816 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:23:32.335025 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.335004 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw" Aug 04 11:23:32.367275 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.367254 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq" Aug 04 11:23:32.435628 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.435599 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e-plugins-config-volume\") pod \"0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e\" (UID: \"0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e\") " Aug 04 11:23:32.435775 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.435635 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/018e4ae9-9fd3-45e0-b211-2183ed1225c3-plugins-config-volume\") pod \"018e4ae9-9fd3-45e0-b211-2183ed1225c3\" (UID: \"018e4ae9-9fd3-45e0-b211-2183ed1225c3\") " Aug 04 11:23:32.435775 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.435662 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvxr9\" (UniqueName: \"kubernetes.io/projected/0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e-kube-api-access-vvxr9\") pod \"0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e\" (UID: \"0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e\") " Aug 04 11:23:32.435775 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.435714 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-75v7z\" (UniqueName: \"kubernetes.io/projected/018e4ae9-9fd3-45e0-b211-2183ed1225c3-kube-api-access-75v7z\") pod \"018e4ae9-9fd3-45e0-b211-2183ed1225c3\" (UID: \"018e4ae9-9fd3-45e0-b211-2183ed1225c3\") " Aug 04 11:23:32.436043 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.436014 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e" (UID: "0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:23:32.436103 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.436020 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/018e4ae9-9fd3-45e0-b211-2183ed1225c3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "018e4ae9-9fd3-45e0-b211-2183ed1225c3" (UID: "018e4ae9-9fd3-45e0-b211-2183ed1225c3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:23:32.437794 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.437762 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e-kube-api-access-vvxr9" (OuterVolumeSpecName: "kube-api-access-vvxr9") pod "0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e" (UID: "0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e"). InnerVolumeSpecName "kube-api-access-vvxr9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 11:23:32.437911 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.437826 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/018e4ae9-9fd3-45e0-b211-2183ed1225c3-kube-api-access-75v7z" (OuterVolumeSpecName: "kube-api-access-75v7z") pod "018e4ae9-9fd3-45e0-b211-2183ed1225c3" (UID: "018e4ae9-9fd3-45e0-b211-2183ed1225c3"). InnerVolumeSpecName "kube-api-access-75v7z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 11:23:32.480748 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.480722 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50a45b17-d14b-4a69-be3b-6c73cd6d656f" path="/var/lib/kubelet/pods/50a45b17-d14b-4a69-be3b-6c73cd6d656f/volumes" Aug 04 11:23:32.537300 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.537247 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:23:32.537300 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.537269 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/018e4ae9-9fd3-45e0-b211-2183ed1225c3-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:23:32.537300 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.537280 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvxr9\" (UniqueName: \"kubernetes.io/projected/0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e-kube-api-access-vvxr9\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:23:32.537300 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.537288 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75v7z\" (UniqueName: \"kubernetes.io/projected/018e4ae9-9fd3-45e0-b211-2183ed1225c3-kube-api-access-75v7z\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:23:32.592952 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.592889 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:23:32.849378 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.849296 2535 generic.go:356] "Generic (PLEG): container finished" podID="018e4ae9-9fd3-45e0-b211-2183ed1225c3" containerID="126f43a12bf1f7015cb2e8a73aa4e89f5d6c6ff99c1c6a57be1a9fe6c7bb1f7a" exitCode=0 Aug 04 11:23:32.849378 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.849342 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw" Aug 04 11:23:32.849616 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.849384 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw" event={"ID":"018e4ae9-9fd3-45e0-b211-2183ed1225c3","Type":"ContainerDied","Data":"126f43a12bf1f7015cb2e8a73aa4e89f5d6c6ff99c1c6a57be1a9fe6c7bb1f7a"} Aug 04 11:23:32.849616 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.849437 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw" event={"ID":"018e4ae9-9fd3-45e0-b211-2183ed1225c3","Type":"ContainerDied","Data":"fa4d79c7607d270d27c62ceb28977a544ef6072ee16d624f3e819cddcf281673"} Aug 04 11:23:32.849616 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.849464 2535 scope.go:117] "RemoveContainer" containerID="126f43a12bf1f7015cb2e8a73aa4e89f5d6c6ff99c1c6a57be1a9fe6c7bb1f7a" Aug 04 11:23:32.850746 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.850722 2535 generic.go:356] "Generic (PLEG): container finished" podID="0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e" containerID="02500e6d072a5214f6047e30c4f402bf949ff36357f05d9204fa94ee03791f33" exitCode=0 Aug 04 11:23:32.850878 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.850776 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq" Aug 04 11:23:32.850878 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.850794 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq" event={"ID":"0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e","Type":"ContainerDied","Data":"02500e6d072a5214f6047e30c4f402bf949ff36357f05d9204fa94ee03791f33"} Aug 04 11:23:32.850878 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.850836 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq" event={"ID":"0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e","Type":"ContainerDied","Data":"13d24d1e50650792890716f749742ef3b8367e47f13970dd66a65a9886f14a4b"} Aug 04 11:23:32.861550 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.861530 2535 scope.go:117] "RemoveContainer" containerID="126f43a12bf1f7015cb2e8a73aa4e89f5d6c6ff99c1c6a57be1a9fe6c7bb1f7a" Aug 04 11:23:32.861831 ip-10-0-130-250 kubenswrapper[2535]: E0804 11:23:32.861812 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"126f43a12bf1f7015cb2e8a73aa4e89f5d6c6ff99c1c6a57be1a9fe6c7bb1f7a\": container with ID starting with 126f43a12bf1f7015cb2e8a73aa4e89f5d6c6ff99c1c6a57be1a9fe6c7bb1f7a not found: ID does not exist" containerID="126f43a12bf1f7015cb2e8a73aa4e89f5d6c6ff99c1c6a57be1a9fe6c7bb1f7a" Aug 04 11:23:32.861887 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.861840 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"126f43a12bf1f7015cb2e8a73aa4e89f5d6c6ff99c1c6a57be1a9fe6c7bb1f7a"} err="failed to get container status \"126f43a12bf1f7015cb2e8a73aa4e89f5d6c6ff99c1c6a57be1a9fe6c7bb1f7a\": rpc error: code = NotFound desc = could not find container \"126f43a12bf1f7015cb2e8a73aa4e89f5d6c6ff99c1c6a57be1a9fe6c7bb1f7a\": container with ID starting with 126f43a12bf1f7015cb2e8a73aa4e89f5d6c6ff99c1c6a57be1a9fe6c7bb1f7a not found: ID does not exist" Aug 04 11:23:32.861887 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.861855 2535 scope.go:117] "RemoveContainer" containerID="02500e6d072a5214f6047e30c4f402bf949ff36357f05d9204fa94ee03791f33" Aug 04 11:23:32.869071 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.869050 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw"] Aug 04 11:23:32.871123 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.871103 2535 scope.go:117] "RemoveContainer" containerID="02500e6d072a5214f6047e30c4f402bf949ff36357f05d9204fa94ee03791f33" Aug 04 11:23:32.871418 ip-10-0-130-250 kubenswrapper[2535]: E0804 11:23:32.871396 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02500e6d072a5214f6047e30c4f402bf949ff36357f05d9204fa94ee03791f33\": container with ID starting with 02500e6d072a5214f6047e30c4f402bf949ff36357f05d9204fa94ee03791f33 not found: ID does not exist" containerID="02500e6d072a5214f6047e30c4f402bf949ff36357f05d9204fa94ee03791f33" Aug 04 11:23:32.871521 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.871423 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02500e6d072a5214f6047e30c4f402bf949ff36357f05d9204fa94ee03791f33"} err="failed to get container status \"02500e6d072a5214f6047e30c4f402bf949ff36357f05d9204fa94ee03791f33\": rpc error: code = NotFound desc = could not find container \"02500e6d072a5214f6047e30c4f402bf949ff36357f05d9204fa94ee03791f33\": container with ID starting with 02500e6d072a5214f6047e30c4f402bf949ff36357f05d9204fa94ee03791f33 not found: ID does not exist" Aug 04 11:23:32.872778 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.872758 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-a031b5-c74bbf67b-zwqrw"] Aug 04 11:23:32.881685 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.881664 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq"] Aug 04 11:23:32.884747 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:32.884726 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a031b5-bd5986b6d-pt4mq"] Aug 04 11:23:34.479819 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:34.479784 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="018e4ae9-9fd3-45e0-b211-2183ed1225c3" path="/var/lib/kubelet/pods/018e4ae9-9fd3-45e0-b211-2183ed1225c3/volumes" Aug 04 11:23:34.480196 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:34.480144 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e" path="/var/lib/kubelet/pods/0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e/volumes" Aug 04 11:23:38.918895 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:38.918857 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:23:39.021713 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:39.021679 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:23:39.194507 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:39.194396 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:23:39.433497 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:39.433467 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:23:42.940973 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:42.940938 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1cb57d-44vpb"] Aug 04 11:23:42.941354 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:42.941341 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e" containerName="payload-pre-processing" Aug 04 11:23:42.941398 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:42.941355 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e" containerName="payload-pre-processing" Aug 04 11:23:42.941398 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:42.941364 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="018e4ae9-9fd3-45e0-b211-2183ed1225c3" containerName="payload-processing" Aug 04 11:23:42.941398 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:42.941370 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="018e4ae9-9fd3-45e0-b211-2183ed1225c3" containerName="payload-processing" Aug 04 11:23:42.941398 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:42.941384 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50a45b17-d14b-4a69-be3b-6c73cd6d656f" containerName="revoke-keys" Aug 04 11:23:42.941398 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:42.941390 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="50a45b17-d14b-4a69-be3b-6c73cd6d656f" containerName="revoke-keys" Aug 04 11:23:42.941570 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:42.941447 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="018e4ae9-9fd3-45e0-b211-2183ed1225c3" containerName="payload-processing" Aug 04 11:23:42.941570 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:42.941456 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ef4c3fb-fd83-4789-be8b-bd5d4964ba6e" containerName="payload-pre-processing" Aug 04 11:23:42.941570 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:42.941465 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="50a45b17-d14b-4a69-be3b-6c73cd6d656f" containerName="revoke-keys" Aug 04 11:23:42.945745 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:42.945726 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1cb57d-44vpb" Aug 04 11:23:42.948228 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:42.948201 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qptbt\"" Aug 04 11:23:42.951264 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:42.951224 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1cb57d-44vpb"] Aug 04 11:23:43.039273 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:43.039236 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41378122-8191-4a32-9a12-0e73b8b1f6ef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-1cb57d-44vpb\" (UID: \"41378122-8191-4a32-9a12-0e73b8b1f6ef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1cb57d-44vpb" Aug 04 11:23:43.140382 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:43.140297 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41378122-8191-4a32-9a12-0e73b8b1f6ef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-1cb57d-44vpb\" (UID: \"41378122-8191-4a32-9a12-0e73b8b1f6ef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1cb57d-44vpb" Aug 04 11:23:43.140945 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:43.140923 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41378122-8191-4a32-9a12-0e73b8b1f6ef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-1cb57d-44vpb\" (UID: \"41378122-8191-4a32-9a12-0e73b8b1f6ef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1cb57d-44vpb" Aug 04 11:23:43.259106 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:43.259065 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1cb57d-44vpb" Aug 04 11:23:43.605697 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:43.605664 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1cb57d-44vpb"] Aug 04 11:23:43.607410 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:23:43.607386 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41378122_8191_4a32_9a12_0e73b8b1f6ef.slice/crio-a68b1cd8bbd6efc6b115e686beb2944eda0417fa31a08f72fe8d56f0f392a4a3 WatchSource:0}: Error finding container a68b1cd8bbd6efc6b115e686beb2944eda0417fa31a08f72fe8d56f0f392a4a3: Status 404 returned error can't find the container with id a68b1cd8bbd6efc6b115e686beb2944eda0417fa31a08f72fe8d56f0f392a4a3 Aug 04 11:23:43.910891 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:43.910800 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1cb57d-44vpb" event={"ID":"41378122-8191-4a32-9a12-0e73b8b1f6ef","Type":"ContainerStarted","Data":"d1df7c2381479654c3e3ce313ee9a37882826de435ca609c1541c035067e914b"} Aug 04 11:23:43.910891 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:43.910867 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1cb57d-44vpb" event={"ID":"41378122-8191-4a32-9a12-0e73b8b1f6ef","Type":"ContainerStarted","Data":"a68b1cd8bbd6efc6b115e686beb2944eda0417fa31a08f72fe8d56f0f392a4a3"} Aug 04 11:23:43.926131 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:43.926072 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1cb57d-44vpb" podStartSLOduration=1.926052435 podStartE2EDuration="1.926052435s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 11:23:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:23:43.925092555 +0000 UTC m=+3361.899175096" watchObservedRunningTime="2026-08-04 11:23:43.926052435 +0000 UTC m=+3361.900134964" Aug 04 11:23:44.917054 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:44.917015 2535 generic.go:356] "Generic (PLEG): container finished" podID="41378122-8191-4a32-9a12-0e73b8b1f6ef" containerID="d1df7c2381479654c3e3ce313ee9a37882826de435ca609c1541c035067e914b" exitCode=0 Aug 04 11:23:44.917473 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:44.917099 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1cb57d-44vpb" event={"ID":"41378122-8191-4a32-9a12-0e73b8b1f6ef","Type":"ContainerDied","Data":"d1df7c2381479654c3e3ce313ee9a37882826de435ca609c1541c035067e914b"} Aug 04 11:23:46.059934 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:46.059908 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1cb57d-44vpb" Aug 04 11:23:46.166371 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:46.166336 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41378122-8191-4a32-9a12-0e73b8b1f6ef-maas-api-service-ca\") pod \"41378122-8191-4a32-9a12-0e73b8b1f6ef\" (UID: \"41378122-8191-4a32-9a12-0e73b8b1f6ef\") " Aug 04 11:23:46.166682 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:46.166655 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41378122-8191-4a32-9a12-0e73b8b1f6ef-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "41378122-8191-4a32-9a12-0e73b8b1f6ef" (UID: "41378122-8191-4a32-9a12-0e73b8b1f6ef"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:23:46.268205 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:46.268124 2535 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41378122-8191-4a32-9a12-0e73b8b1f6ef-maas-api-service-ca\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:23:46.928430 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:46.928398 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1cb57d-44vpb" Aug 04 11:23:46.928629 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:46.928428 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1cb57d-44vpb" event={"ID":"41378122-8191-4a32-9a12-0e73b8b1f6ef","Type":"ContainerDied","Data":"a68b1cd8bbd6efc6b115e686beb2944eda0417fa31a08f72fe8d56f0f392a4a3"} Aug 04 11:23:46.928629 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:46.928502 2535 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a68b1cd8bbd6efc6b115e686beb2944eda0417fa31a08f72fe8d56f0f392a4a3" Aug 04 11:23:58.433610 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.433577 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:23:58.453398 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.453365 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw"] Aug 04 11:23:58.453621 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.453597 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw" podUID="9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a" containerName="payload-processing" containerID="cri-o://deb9ec9cb649c68bf7d2365708c8fd1d488f90a0b41b4db6aa026e366a1519b5" gracePeriod=30 Aug 04 11:23:58.459966 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.459385 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7"] Aug 04 11:23:58.459966 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.459624 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7" podUID="ca6c79c0-045c-4d38-a3eb-4a4c80adb97a" containerName="payload-pre-processing" containerID="cri-o://0638a431768d8753daeef11ca7d5869c3dc7d93eb8bd59936602a489ed79c58f" gracePeriod=30 Aug 04 11:23:58.499448 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.499421 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:23:58.602698 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.602616 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:23:58.612627 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.612598 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1cb57d-44vpb"] Aug 04 11:23:58.621914 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.619799 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-1cb57d-44vpb"] Aug 04 11:23:58.690138 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.690064 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:23:58.744676 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.744652 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw" Aug 04 11:23:58.747529 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.747511 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7" Aug 04 11:23:58.787619 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.787589 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcxd4\" (UniqueName: \"kubernetes.io/projected/9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a-kube-api-access-tcxd4\") pod \"9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a\" (UID: \"9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a\") " Aug 04 11:23:58.788678 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.787645 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nj8wg\" (UniqueName: \"kubernetes.io/projected/ca6c79c0-045c-4d38-a3eb-4a4c80adb97a-kube-api-access-nj8wg\") pod \"ca6c79c0-045c-4d38-a3eb-4a4c80adb97a\" (UID: \"ca6c79c0-045c-4d38-a3eb-4a4c80adb97a\") " Aug 04 11:23:58.788678 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.787689 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a-plugins-config-volume\") pod \"9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a\" (UID: \"9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a\") " Aug 04 11:23:58.788678 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.787751 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca6c79c0-045c-4d38-a3eb-4a4c80adb97a-plugins-config-volume\") pod \"ca6c79c0-045c-4d38-a3eb-4a4c80adb97a\" (UID: \"ca6c79c0-045c-4d38-a3eb-4a4c80adb97a\") " Aug 04 11:23:58.788678 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.788196 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a" (UID: "9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:23:58.788678 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.788471 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca6c79c0-045c-4d38-a3eb-4a4c80adb97a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ca6c79c0-045c-4d38-a3eb-4a4c80adb97a" (UID: "ca6c79c0-045c-4d38-a3eb-4a4c80adb97a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:23:58.791822 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.791788 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca6c79c0-045c-4d38-a3eb-4a4c80adb97a-kube-api-access-nj8wg" (OuterVolumeSpecName: "kube-api-access-nj8wg") pod "ca6c79c0-045c-4d38-a3eb-4a4c80adb97a" (UID: "ca6c79c0-045c-4d38-a3eb-4a4c80adb97a"). InnerVolumeSpecName "kube-api-access-nj8wg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 11:23:58.791951 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.791815 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a-kube-api-access-tcxd4" (OuterVolumeSpecName: "kube-api-access-tcxd4") pod "9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a" (UID: "9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a"). InnerVolumeSpecName "kube-api-access-tcxd4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 11:23:58.889106 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.889076 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nj8wg\" (UniqueName: \"kubernetes.io/projected/ca6c79c0-045c-4d38-a3eb-4a4c80adb97a-kube-api-access-nj8wg\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:23:58.889106 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.889102 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:23:58.889348 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.889116 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca6c79c0-045c-4d38-a3eb-4a4c80adb97a-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:23:58.889348 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.889128 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcxd4\" (UniqueName: \"kubernetes.io/projected/9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a-kube-api-access-tcxd4\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:23:58.990625 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.990547 2535 generic.go:356] "Generic (PLEG): container finished" podID="ca6c79c0-045c-4d38-a3eb-4a4c80adb97a" containerID="0638a431768d8753daeef11ca7d5869c3dc7d93eb8bd59936602a489ed79c58f" exitCode=0 Aug 04 11:23:58.990625 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.990591 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7" Aug 04 11:23:58.990825 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.990635 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7" event={"ID":"ca6c79c0-045c-4d38-a3eb-4a4c80adb97a","Type":"ContainerDied","Data":"0638a431768d8753daeef11ca7d5869c3dc7d93eb8bd59936602a489ed79c58f"} Aug 04 11:23:58.990825 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.990693 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7" event={"ID":"ca6c79c0-045c-4d38-a3eb-4a4c80adb97a","Type":"ContainerDied","Data":"c2c4a3e88d8f911c2743ddebf97b57b5d4bc7d051062f25472809dab5b3bca6e"} Aug 04 11:23:58.990825 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.990716 2535 scope.go:117] "RemoveContainer" containerID="0638a431768d8753daeef11ca7d5869c3dc7d93eb8bd59936602a489ed79c58f" Aug 04 11:23:58.992024 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.992000 2535 generic.go:356] "Generic (PLEG): container finished" podID="9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a" containerID="deb9ec9cb649c68bf7d2365708c8fd1d488f90a0b41b4db6aa026e366a1519b5" exitCode=0 Aug 04 11:23:58.992144 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.992027 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw" event={"ID":"9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a","Type":"ContainerDied","Data":"deb9ec9cb649c68bf7d2365708c8fd1d488f90a0b41b4db6aa026e366a1519b5"} Aug 04 11:23:58.992144 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.992045 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw" Aug 04 11:23:58.992144 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:58.992086 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw" event={"ID":"9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a","Type":"ContainerDied","Data":"2a1f5c4ccee86162059e2a57675dd6b51bcd4a888ad999aac9a1c5bd83790ff1"} Aug 04 11:23:59.001349 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:59.001332 2535 scope.go:117] "RemoveContainer" containerID="0638a431768d8753daeef11ca7d5869c3dc7d93eb8bd59936602a489ed79c58f" Aug 04 11:23:59.001656 ip-10-0-130-250 kubenswrapper[2535]: E0804 11:23:59.001637 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0638a431768d8753daeef11ca7d5869c3dc7d93eb8bd59936602a489ed79c58f\": container with ID starting with 0638a431768d8753daeef11ca7d5869c3dc7d93eb8bd59936602a489ed79c58f not found: ID does not exist" containerID="0638a431768d8753daeef11ca7d5869c3dc7d93eb8bd59936602a489ed79c58f" Aug 04 11:23:59.001765 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:59.001662 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0638a431768d8753daeef11ca7d5869c3dc7d93eb8bd59936602a489ed79c58f"} err="failed to get container status \"0638a431768d8753daeef11ca7d5869c3dc7d93eb8bd59936602a489ed79c58f\": rpc error: code = NotFound desc = could not find container \"0638a431768d8753daeef11ca7d5869c3dc7d93eb8bd59936602a489ed79c58f\": container with ID starting with 0638a431768d8753daeef11ca7d5869c3dc7d93eb8bd59936602a489ed79c58f not found: ID does not exist" Aug 04 11:23:59.001765 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:59.001677 2535 scope.go:117] "RemoveContainer" containerID="deb9ec9cb649c68bf7d2365708c8fd1d488f90a0b41b4db6aa026e366a1519b5" Aug 04 11:23:59.009535 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:59.009515 2535 scope.go:117] "RemoveContainer" containerID="deb9ec9cb649c68bf7d2365708c8fd1d488f90a0b41b4db6aa026e366a1519b5" Aug 04 11:23:59.009786 ip-10-0-130-250 kubenswrapper[2535]: E0804 11:23:59.009769 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"deb9ec9cb649c68bf7d2365708c8fd1d488f90a0b41b4db6aa026e366a1519b5\": container with ID starting with deb9ec9cb649c68bf7d2365708c8fd1d488f90a0b41b4db6aa026e366a1519b5 not found: ID does not exist" containerID="deb9ec9cb649c68bf7d2365708c8fd1d488f90a0b41b4db6aa026e366a1519b5" Aug 04 11:23:59.009862 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:59.009792 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"deb9ec9cb649c68bf7d2365708c8fd1d488f90a0b41b4db6aa026e366a1519b5"} err="failed to get container status \"deb9ec9cb649c68bf7d2365708c8fd1d488f90a0b41b4db6aa026e366a1519b5\": rpc error: code = NotFound desc = could not find container \"deb9ec9cb649c68bf7d2365708c8fd1d488f90a0b41b4db6aa026e366a1519b5\": container with ID starting with deb9ec9cb649c68bf7d2365708c8fd1d488f90a0b41b4db6aa026e366a1519b5 not found: ID does not exist" Aug 04 11:23:59.013673 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:59.013652 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7"] Aug 04 11:23:59.018751 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:59.018731 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-1cb57d-7c486848d-d6ls7"] Aug 04 11:23:59.028748 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:59.028730 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:23:59.033753 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:59.033733 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw"] Aug 04 11:23:59.038519 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:23:59.038499 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-1cb57d-66c488b978-fpqlw"] Aug 04 11:24:00.480914 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:00.480882 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41378122-8191-4a32-9a12-0e73b8b1f6ef" path="/var/lib/kubelet/pods/41378122-8191-4a32-9a12-0e73b8b1f6ef/volumes" Aug 04 11:24:00.481280 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:00.481219 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a" path="/var/lib/kubelet/pods/9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a/volumes" Aug 04 11:24:00.481541 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:00.481528 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca6c79c0-045c-4d38-a3eb-4a4c80adb97a" path="/var/lib/kubelet/pods/ca6c79c0-045c-4d38-a3eb-4a4c80adb97a/volumes" Aug 04 11:24:04.014753 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:04.014716 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:24:40.809679 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:40.809598 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:24:42.042691 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:42.042656 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:24:48.051357 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:48.051320 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:24:48.166103 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:48.166070 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:24:48.345748 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:48.345652 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:24:48.554179 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:48.554131 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:24:48.681114 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:48.681025 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-651a7a-r52sz"] Aug 04 11:24:48.681443 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:48.681429 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca6c79c0-045c-4d38-a3eb-4a4c80adb97a" containerName="payload-pre-processing" Aug 04 11:24:48.681443 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:48.681444 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca6c79c0-045c-4d38-a3eb-4a4c80adb97a" containerName="payload-pre-processing" Aug 04 11:24:48.681601 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:48.681458 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41378122-8191-4a32-9a12-0e73b8b1f6ef" containerName="revoke-keys" Aug 04 11:24:48.681601 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:48.681465 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="41378122-8191-4a32-9a12-0e73b8b1f6ef" containerName="revoke-keys" Aug 04 11:24:48.681601 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:48.681479 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a" containerName="payload-processing" Aug 04 11:24:48.681601 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:48.681500 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a" containerName="payload-processing" Aug 04 11:24:48.681601 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:48.681564 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="41378122-8191-4a32-9a12-0e73b8b1f6ef" containerName="revoke-keys" Aug 04 11:24:48.681601 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:48.681573 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cf3792d-8f3e-4bf1-8a23-161ed41b8e1a" containerName="payload-processing" Aug 04 11:24:48.681601 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:48.681582 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca6c79c0-045c-4d38-a3eb-4a4c80adb97a" containerName="payload-pre-processing" Aug 04 11:24:48.686291 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:48.686269 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-651a7a-r52sz" Aug 04 11:24:48.688315 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:48.688292 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qptbt\"" Aug 04 11:24:48.693116 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:48.693096 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-651a7a-r52sz"] Aug 04 11:24:48.840762 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:48.840732 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d3a4943-2e54-4289-9a3c-e291d8918195-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-651a7a-r52sz\" (UID: \"0d3a4943-2e54-4289-9a3c-e291d8918195\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-651a7a-r52sz" Aug 04 11:24:48.942523 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:48.942429 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d3a4943-2e54-4289-9a3c-e291d8918195-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-651a7a-r52sz\" (UID: \"0d3a4943-2e54-4289-9a3c-e291d8918195\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-651a7a-r52sz" Aug 04 11:24:48.943044 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:48.943023 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d3a4943-2e54-4289-9a3c-e291d8918195-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-651a7a-r52sz\" (UID: \"0d3a4943-2e54-4289-9a3c-e291d8918195\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-651a7a-r52sz" Aug 04 11:24:48.998959 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:48.998930 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-651a7a-r52sz" Aug 04 11:24:49.132951 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:49.132927 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-651a7a-r52sz"] Aug 04 11:24:49.135696 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:24:49.135668 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d3a4943_2e54_4289_9a3c_e291d8918195.slice/crio-65113f31cb9e17663ed87fdc0faddc3169f3c5d74feb544071318a3193cd7b3d WatchSource:0}: Error finding container 65113f31cb9e17663ed87fdc0faddc3169f3c5d74feb544071318a3193cd7b3d: Status 404 returned error can't find the container with id 65113f31cb9e17663ed87fdc0faddc3169f3c5d74feb544071318a3193cd7b3d Aug 04 11:24:49.250298 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:49.250218 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-651a7a-r52sz" event={"ID":"0d3a4943-2e54-4289-9a3c-e291d8918195","Type":"ContainerStarted","Data":"65113f31cb9e17663ed87fdc0faddc3169f3c5d74feb544071318a3193cd7b3d"} Aug 04 11:24:50.257375 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:50.257294 2535 generic.go:356] "Generic (PLEG): container finished" podID="0d3a4943-2e54-4289-9a3c-e291d8918195" containerID="7f91cf8b6220bb60a92d43615a27bf2f819bf84bd5af0ac5146dd6d8afbbeadc" exitCode=0 Aug 04 11:24:50.257375 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:50.257332 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-651a7a-r52sz" event={"ID":"0d3a4943-2e54-4289-9a3c-e291d8918195","Type":"ContainerDied","Data":"7f91cf8b6220bb60a92d43615a27bf2f819bf84bd5af0ac5146dd6d8afbbeadc"} Aug 04 11:24:51.406313 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:51.406285 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-651a7a-r52sz" Aug 04 11:24:51.566972 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:51.566941 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d3a4943-2e54-4289-9a3c-e291d8918195-maas-api-service-ca\") pod \"0d3a4943-2e54-4289-9a3c-e291d8918195\" (UID: \"0d3a4943-2e54-4289-9a3c-e291d8918195\") " Aug 04 11:24:51.567311 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:51.567289 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d3a4943-2e54-4289-9a3c-e291d8918195-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0d3a4943-2e54-4289-9a3c-e291d8918195" (UID: "0d3a4943-2e54-4289-9a3c-e291d8918195"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:24:51.667751 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:51.667722 2535 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d3a4943-2e54-4289-9a3c-e291d8918195-maas-api-service-ca\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:24:52.268531 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:52.268482 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-651a7a-r52sz" Aug 04 11:24:52.268706 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:52.268525 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-651a7a-r52sz" event={"ID":"0d3a4943-2e54-4289-9a3c-e291d8918195","Type":"ContainerDied","Data":"65113f31cb9e17663ed87fdc0faddc3169f3c5d74feb544071318a3193cd7b3d"} Aug 04 11:24:52.268706 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:52.268574 2535 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="65113f31cb9e17663ed87fdc0faddc3169f3c5d74feb544071318a3193cd7b3d" Aug 04 11:24:57.649096 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:57.649026 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:24:59.175837 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:59.175796 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:24:59.209077 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:59.209021 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s"] Aug 04 11:24:59.209341 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:59.209312 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s" podUID="263ebf50-1233-463f-9c5b-d021021ad60d" containerName="payload-processing" containerID="cri-o://f814380822f158215b424ee674c7210f9c9c2bc7c53c3ad6987b1b88cfc28572" gracePeriod=30 Aug 04 11:24:59.218349 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:59.218316 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5"] Aug 04 11:24:59.218692 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:59.218637 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5" podUID="1ca9eaa7-e647-46c8-8474-748a69e4a737" containerName="payload-pre-processing" containerID="cri-o://8b0a023a9c5b66e0b267e3037de76461c0b1aa326b5225fb1c2d248c383a2dbe" gracePeriod=30 Aug 04 11:24:59.253611 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:59.253572 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:24:59.335162 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:59.335108 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:24:59.411074 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:59.411044 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-651a7a-r52sz"] Aug 04 11:24:59.424049 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:59.423939 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-651a7a-r52sz"] Aug 04 11:24:59.424359 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:59.424326 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:24:59.471459 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:59.471437 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s" Aug 04 11:24:59.535812 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:59.535791 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5" Aug 04 11:24:59.642271 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:59.642241 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82hm9\" (UniqueName: \"kubernetes.io/projected/1ca9eaa7-e647-46c8-8474-748a69e4a737-kube-api-access-82hm9\") pod \"1ca9eaa7-e647-46c8-8474-748a69e4a737\" (UID: \"1ca9eaa7-e647-46c8-8474-748a69e4a737\") " Aug 04 11:24:59.642407 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:59.642291 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ca9eaa7-e647-46c8-8474-748a69e4a737-plugins-config-volume\") pod \"1ca9eaa7-e647-46c8-8474-748a69e4a737\" (UID: \"1ca9eaa7-e647-46c8-8474-748a69e4a737\") " Aug 04 11:24:59.642407 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:59.642317 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qkg5\" (UniqueName: \"kubernetes.io/projected/263ebf50-1233-463f-9c5b-d021021ad60d-kube-api-access-5qkg5\") pod \"263ebf50-1233-463f-9c5b-d021021ad60d\" (UID: \"263ebf50-1233-463f-9c5b-d021021ad60d\") " Aug 04 11:24:59.642407 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:59.642387 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/263ebf50-1233-463f-9c5b-d021021ad60d-plugins-config-volume\") pod \"263ebf50-1233-463f-9c5b-d021021ad60d\" (UID: \"263ebf50-1233-463f-9c5b-d021021ad60d\") " Aug 04 11:24:59.642728 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:59.642704 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ca9eaa7-e647-46c8-8474-748a69e4a737-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1ca9eaa7-e647-46c8-8474-748a69e4a737" (UID: "1ca9eaa7-e647-46c8-8474-748a69e4a737"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:24:59.642825 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:59.642776 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/263ebf50-1233-463f-9c5b-d021021ad60d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "263ebf50-1233-463f-9c5b-d021021ad60d" (UID: "263ebf50-1233-463f-9c5b-d021021ad60d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:24:59.644320 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:59.644300 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/263ebf50-1233-463f-9c5b-d021021ad60d-kube-api-access-5qkg5" (OuterVolumeSpecName: "kube-api-access-5qkg5") pod "263ebf50-1233-463f-9c5b-d021021ad60d" (UID: "263ebf50-1233-463f-9c5b-d021021ad60d"). InnerVolumeSpecName "kube-api-access-5qkg5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 11:24:59.644372 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:59.644342 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ca9eaa7-e647-46c8-8474-748a69e4a737-kube-api-access-82hm9" (OuterVolumeSpecName: "kube-api-access-82hm9") pod "1ca9eaa7-e647-46c8-8474-748a69e4a737" (UID: "1ca9eaa7-e647-46c8-8474-748a69e4a737"). InnerVolumeSpecName "kube-api-access-82hm9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 11:24:59.743292 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:59.743213 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/263ebf50-1233-463f-9c5b-d021021ad60d-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:24:59.743292 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:59.743243 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82hm9\" (UniqueName: \"kubernetes.io/projected/1ca9eaa7-e647-46c8-8474-748a69e4a737-kube-api-access-82hm9\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:24:59.743292 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:59.743253 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ca9eaa7-e647-46c8-8474-748a69e4a737-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:24:59.743292 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:59.743261 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qkg5\" (UniqueName: \"kubernetes.io/projected/263ebf50-1233-463f-9c5b-d021021ad60d-kube-api-access-5qkg5\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:24:59.783533 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:24:59.783501 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:25:00.314775 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:00.314739 2535 generic.go:356] "Generic (PLEG): container finished" podID="263ebf50-1233-463f-9c5b-d021021ad60d" containerID="f814380822f158215b424ee674c7210f9c9c2bc7c53c3ad6987b1b88cfc28572" exitCode=0 Aug 04 11:25:00.315234 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:00.314780 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s" Aug 04 11:25:00.315234 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:00.314822 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s" event={"ID":"263ebf50-1233-463f-9c5b-d021021ad60d","Type":"ContainerDied","Data":"f814380822f158215b424ee674c7210f9c9c2bc7c53c3ad6987b1b88cfc28572"} Aug 04 11:25:00.315234 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:00.314887 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s" event={"ID":"263ebf50-1233-463f-9c5b-d021021ad60d","Type":"ContainerDied","Data":"e7be6ddb12751b1aa4c019a40feb77c7fc3f1b6d9e7b81bb88334a8b2943e450"} Aug 04 11:25:00.315234 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:00.314923 2535 scope.go:117] "RemoveContainer" containerID="f814380822f158215b424ee674c7210f9c9c2bc7c53c3ad6987b1b88cfc28572" Aug 04 11:25:00.316316 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:00.316297 2535 generic.go:356] "Generic (PLEG): container finished" podID="1ca9eaa7-e647-46c8-8474-748a69e4a737" containerID="8b0a023a9c5b66e0b267e3037de76461c0b1aa326b5225fb1c2d248c383a2dbe" exitCode=0 Aug 04 11:25:00.316400 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:00.316344 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5" Aug 04 11:25:00.316400 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:00.316386 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5" event={"ID":"1ca9eaa7-e647-46c8-8474-748a69e4a737","Type":"ContainerDied","Data":"8b0a023a9c5b66e0b267e3037de76461c0b1aa326b5225fb1c2d248c383a2dbe"} Aug 04 11:25:00.316476 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:00.316439 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5" event={"ID":"1ca9eaa7-e647-46c8-8474-748a69e4a737","Type":"ContainerDied","Data":"08fd7635f0f659085bfc35ba899643862a1e6c56a6b98ac149b18e82fc2e9708"} Aug 04 11:25:00.326347 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:00.326332 2535 scope.go:117] "RemoveContainer" containerID="f814380822f158215b424ee674c7210f9c9c2bc7c53c3ad6987b1b88cfc28572" Aug 04 11:25:00.326607 ip-10-0-130-250 kubenswrapper[2535]: E0804 11:25:00.326587 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f814380822f158215b424ee674c7210f9c9c2bc7c53c3ad6987b1b88cfc28572\": container with ID starting with f814380822f158215b424ee674c7210f9c9c2bc7c53c3ad6987b1b88cfc28572 not found: ID does not exist" containerID="f814380822f158215b424ee674c7210f9c9c2bc7c53c3ad6987b1b88cfc28572" Aug 04 11:25:00.326678 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:00.326614 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f814380822f158215b424ee674c7210f9c9c2bc7c53c3ad6987b1b88cfc28572"} err="failed to get container status \"f814380822f158215b424ee674c7210f9c9c2bc7c53c3ad6987b1b88cfc28572\": rpc error: code = NotFound desc = could not find container \"f814380822f158215b424ee674c7210f9c9c2bc7c53c3ad6987b1b88cfc28572\": container with ID starting with f814380822f158215b424ee674c7210f9c9c2bc7c53c3ad6987b1b88cfc28572 not found: ID does not exist" Aug 04 11:25:00.326678 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:00.326632 2535 scope.go:117] "RemoveContainer" containerID="8b0a023a9c5b66e0b267e3037de76461c0b1aa326b5225fb1c2d248c383a2dbe" Aug 04 11:25:00.334601 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:00.334581 2535 scope.go:117] "RemoveContainer" containerID="8b0a023a9c5b66e0b267e3037de76461c0b1aa326b5225fb1c2d248c383a2dbe" Aug 04 11:25:00.334860 ip-10-0-130-250 kubenswrapper[2535]: E0804 11:25:00.334841 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b0a023a9c5b66e0b267e3037de76461c0b1aa326b5225fb1c2d248c383a2dbe\": container with ID starting with 8b0a023a9c5b66e0b267e3037de76461c0b1aa326b5225fb1c2d248c383a2dbe not found: ID does not exist" containerID="8b0a023a9c5b66e0b267e3037de76461c0b1aa326b5225fb1c2d248c383a2dbe" Aug 04 11:25:00.334912 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:00.334869 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b0a023a9c5b66e0b267e3037de76461c0b1aa326b5225fb1c2d248c383a2dbe"} err="failed to get container status \"8b0a023a9c5b66e0b267e3037de76461c0b1aa326b5225fb1c2d248c383a2dbe\": rpc error: code = NotFound desc = could not find container \"8b0a023a9c5b66e0b267e3037de76461c0b1aa326b5225fb1c2d248c383a2dbe\": container with ID starting with 8b0a023a9c5b66e0b267e3037de76461c0b1aa326b5225fb1c2d248c383a2dbe not found: ID does not exist" Aug 04 11:25:00.340694 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:00.340674 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s"] Aug 04 11:25:00.344733 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:00.344713 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-651a7a-c96f9c6db-skc8s"] Aug 04 11:25:00.355041 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:00.355013 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5"] Aug 04 11:25:00.357413 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:00.357390 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-651a7a-66d767454d-qsnc5"] Aug 04 11:25:00.480587 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:00.480515 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d3a4943-2e54-4289-9a3c-e291d8918195" path="/var/lib/kubelet/pods/0d3a4943-2e54-4289-9a3c-e291d8918195/volumes" Aug 04 11:25:00.480858 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:00.480846 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ca9eaa7-e647-46c8-8474-748a69e4a737" path="/var/lib/kubelet/pods/1ca9eaa7-e647-46c8-8474-748a69e4a737/volumes" Aug 04 11:25:00.481153 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:00.481142 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="263ebf50-1233-463f-9c5b-d021021ad60d" path="/var/lib/kubelet/pods/263ebf50-1233-463f-9c5b-d021021ad60d/volumes" Aug 04 11:25:05.925843 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:05.925806 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:25:05.992567 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:05.992532 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:25:06.305864 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:06.305830 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:25:06.467901 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:06.467872 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:25:06.615730 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:06.615652 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bc4a78-q4z46"] Aug 04 11:25:06.616049 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:06.616035 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d3a4943-2e54-4289-9a3c-e291d8918195" containerName="revoke-keys" Aug 04 11:25:06.616095 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:06.616050 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d3a4943-2e54-4289-9a3c-e291d8918195" containerName="revoke-keys" Aug 04 11:25:06.616095 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:06.616062 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="263ebf50-1233-463f-9c5b-d021021ad60d" containerName="payload-processing" Aug 04 11:25:06.616095 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:06.616067 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="263ebf50-1233-463f-9c5b-d021021ad60d" containerName="payload-processing" Aug 04 11:25:06.616095 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:06.616082 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ca9eaa7-e647-46c8-8474-748a69e4a737" containerName="payload-pre-processing" Aug 04 11:25:06.616095 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:06.616088 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ca9eaa7-e647-46c8-8474-748a69e4a737" containerName="payload-pre-processing" Aug 04 11:25:06.616247 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:06.616150 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="263ebf50-1233-463f-9c5b-d021021ad60d" containerName="payload-processing" Aug 04 11:25:06.616247 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:06.616158 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ca9eaa7-e647-46c8-8474-748a69e4a737" containerName="payload-pre-processing" Aug 04 11:25:06.616247 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:06.616166 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d3a4943-2e54-4289-9a3c-e291d8918195" containerName="revoke-keys" Aug 04 11:25:06.620752 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:06.620734 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bc4a78-q4z46" Aug 04 11:25:06.623085 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:06.623059 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qptbt\"" Aug 04 11:25:06.626020 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:06.625999 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bc4a78-q4z46"] Aug 04 11:25:06.701230 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:06.701196 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f5708b8-549f-40f1-a6e3-44efd51e5dfb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-bc4a78-q4z46\" (UID: \"7f5708b8-549f-40f1-a6e3-44efd51e5dfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bc4a78-q4z46" Aug 04 11:25:06.802387 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:06.802359 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f5708b8-549f-40f1-a6e3-44efd51e5dfb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-bc4a78-q4z46\" (UID: \"7f5708b8-549f-40f1-a6e3-44efd51e5dfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bc4a78-q4z46" Aug 04 11:25:06.802950 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:06.802931 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f5708b8-549f-40f1-a6e3-44efd51e5dfb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-bc4a78-q4z46\" (UID: \"7f5708b8-549f-40f1-a6e3-44efd51e5dfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bc4a78-q4z46" Aug 04 11:25:06.932648 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:06.932563 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bc4a78-q4z46" Aug 04 11:25:07.062677 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:07.062650 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bc4a78-q4z46"] Aug 04 11:25:07.065564 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:25:07.065528 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f5708b8_549f_40f1_a6e3_44efd51e5dfb.slice/crio-080994d562af8d7ec5742ad1502a4a93cf4af3213b63ccb1ce31c1233ffd12ac WatchSource:0}: Error finding container 080994d562af8d7ec5742ad1502a4a93cf4af3213b63ccb1ce31c1233ffd12ac: Status 404 returned error can't find the container with id 080994d562af8d7ec5742ad1502a4a93cf4af3213b63ccb1ce31c1233ffd12ac Aug 04 11:25:07.355325 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:07.355294 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bc4a78-q4z46" event={"ID":"7f5708b8-549f-40f1-a6e3-44efd51e5dfb","Type":"ContainerStarted","Data":"080994d562af8d7ec5742ad1502a4a93cf4af3213b63ccb1ce31c1233ffd12ac"} Aug 04 11:25:08.361541 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:08.361503 2535 generic.go:356] "Generic (PLEG): container finished" podID="7f5708b8-549f-40f1-a6e3-44efd51e5dfb" containerID="2017dab7e8b3e568abd448f2e73de1eae4c365c58bce2274427e74d5c40bc91b" exitCode=0 Aug 04 11:25:08.361942 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:08.361581 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bc4a78-q4z46" event={"ID":"7f5708b8-549f-40f1-a6e3-44efd51e5dfb","Type":"ContainerDied","Data":"2017dab7e8b3e568abd448f2e73de1eae4c365c58bce2274427e74d5c40bc91b"} Aug 04 11:25:09.496013 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:09.495991 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bc4a78-q4z46" Aug 04 11:25:09.522964 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:09.522938 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f5708b8-549f-40f1-a6e3-44efd51e5dfb-maas-api-service-ca\") pod \"7f5708b8-549f-40f1-a6e3-44efd51e5dfb\" (UID: \"7f5708b8-549f-40f1-a6e3-44efd51e5dfb\") " Aug 04 11:25:09.523215 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:09.523196 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f5708b8-549f-40f1-a6e3-44efd51e5dfb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7f5708b8-549f-40f1-a6e3-44efd51e5dfb" (UID: "7f5708b8-549f-40f1-a6e3-44efd51e5dfb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:25:09.523310 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:09.523253 2535 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f5708b8-549f-40f1-a6e3-44efd51e5dfb-maas-api-service-ca\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:25:10.371719 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:10.371687 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bc4a78-q4z46" Aug 04 11:25:10.371881 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:10.371723 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bc4a78-q4z46" event={"ID":"7f5708b8-549f-40f1-a6e3-44efd51e5dfb","Type":"ContainerDied","Data":"080994d562af8d7ec5742ad1502a4a93cf4af3213b63ccb1ce31c1233ffd12ac"} Aug 04 11:25:10.371881 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:10.371769 2535 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="080994d562af8d7ec5742ad1502a4a93cf4af3213b63ccb1ce31c1233ffd12ac" Aug 04 11:25:17.198726 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.198692 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bc4a78-29764035-g5dm2"] Aug 04 11:25:17.200928 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.200900 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-bc4a78-29764035-g5dm2"] Aug 04 11:25:17.231365 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.231336 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb"] Aug 04 11:25:17.231653 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.231589 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb" podUID="a3c410c4-0152-44b8-995c-75bb6e1d6f16" containerName="payload-processing" containerID="cri-o://1f6fdd1450ce6fc3a73f713d535128434172024d6aa5b1c22e9ef9ace2aaa8c3" gracePeriod=30 Aug 04 11:25:17.240387 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.239574 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m"] Aug 04 11:25:17.240387 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.239814 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m" podUID="47026a7c-8e2c-4c7e-ac69-4e027cc868aa" containerName="payload-pre-processing" containerID="cri-o://9d2ae7b661a9340c1dd86b18118b32aba1ef789b1b60d47ef33d4b29046a5c36" gracePeriod=30 Aug 04 11:25:17.263468 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.263434 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:25:17.360765 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.360735 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:25:17.377555 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.377518 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bc4a78-q4z46"] Aug 04 11:25:17.382740 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.382715 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bc4a78-q4z46"] Aug 04 11:25:17.413078 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.411463 2535 generic.go:356] "Generic (PLEG): container finished" podID="a3c410c4-0152-44b8-995c-75bb6e1d6f16" containerID="1f6fdd1450ce6fc3a73f713d535128434172024d6aa5b1c22e9ef9ace2aaa8c3" exitCode=0 Aug 04 11:25:17.413078 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.411629 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb" event={"ID":"a3c410c4-0152-44b8-995c-75bb6e1d6f16","Type":"ContainerDied","Data":"1f6fdd1450ce6fc3a73f713d535128434172024d6aa5b1c22e9ef9ace2aaa8c3"} Aug 04 11:25:17.414154 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.413979 2535 generic.go:356] "Generic (PLEG): container finished" podID="47026a7c-8e2c-4c7e-ac69-4e027cc868aa" containerID="9d2ae7b661a9340c1dd86b18118b32aba1ef789b1b60d47ef33d4b29046a5c36" exitCode=0 Aug 04 11:25:17.414154 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.414088 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m" event={"ID":"47026a7c-8e2c-4c7e-ac69-4e027cc868aa","Type":"ContainerDied","Data":"9d2ae7b661a9340c1dd86b18118b32aba1ef789b1b60d47ef33d4b29046a5c36"} Aug 04 11:25:17.447466 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.447420 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:25:17.510380 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.510359 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb" Aug 04 11:25:17.545706 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.545678 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m" Aug 04 11:25:17.594113 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.594084 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47026a7c-8e2c-4c7e-ac69-4e027cc868aa-plugins-config-volume\") pod \"47026a7c-8e2c-4c7e-ac69-4e027cc868aa\" (UID: \"47026a7c-8e2c-4c7e-ac69-4e027cc868aa\") " Aug 04 11:25:17.594310 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.594139 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8l7kg\" (UniqueName: \"kubernetes.io/projected/a3c410c4-0152-44b8-995c-75bb6e1d6f16-kube-api-access-8l7kg\") pod \"a3c410c4-0152-44b8-995c-75bb6e1d6f16\" (UID: \"a3c410c4-0152-44b8-995c-75bb6e1d6f16\") " Aug 04 11:25:17.594310 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.594164 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkwjt\" (UniqueName: \"kubernetes.io/projected/47026a7c-8e2c-4c7e-ac69-4e027cc868aa-kube-api-access-xkwjt\") pod \"47026a7c-8e2c-4c7e-ac69-4e027cc868aa\" (UID: \"47026a7c-8e2c-4c7e-ac69-4e027cc868aa\") " Aug 04 11:25:17.594310 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.594196 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3c410c4-0152-44b8-995c-75bb6e1d6f16-plugins-config-volume\") pod \"a3c410c4-0152-44b8-995c-75bb6e1d6f16\" (UID: \"a3c410c4-0152-44b8-995c-75bb6e1d6f16\") " Aug 04 11:25:17.594616 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.594587 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47026a7c-8e2c-4c7e-ac69-4e027cc868aa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "47026a7c-8e2c-4c7e-ac69-4e027cc868aa" (UID: "47026a7c-8e2c-4c7e-ac69-4e027cc868aa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:25:17.594692 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.594617 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3c410c4-0152-44b8-995c-75bb6e1d6f16-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a3c410c4-0152-44b8-995c-75bb6e1d6f16" (UID: "a3c410c4-0152-44b8-995c-75bb6e1d6f16"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:25:17.596168 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.596148 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3c410c4-0152-44b8-995c-75bb6e1d6f16-kube-api-access-8l7kg" (OuterVolumeSpecName: "kube-api-access-8l7kg") pod "a3c410c4-0152-44b8-995c-75bb6e1d6f16" (UID: "a3c410c4-0152-44b8-995c-75bb6e1d6f16"). InnerVolumeSpecName "kube-api-access-8l7kg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 11:25:17.596234 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.596184 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47026a7c-8e2c-4c7e-ac69-4e027cc868aa-kube-api-access-xkwjt" (OuterVolumeSpecName: "kube-api-access-xkwjt") pod "47026a7c-8e2c-4c7e-ac69-4e027cc868aa" (UID: "47026a7c-8e2c-4c7e-ac69-4e027cc868aa"). InnerVolumeSpecName "kube-api-access-xkwjt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 11:25:17.694863 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.694835 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3c410c4-0152-44b8-995c-75bb6e1d6f16-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:25:17.694863 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.694860 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47026a7c-8e2c-4c7e-ac69-4e027cc868aa-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:25:17.694863 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.694869 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8l7kg\" (UniqueName: \"kubernetes.io/projected/a3c410c4-0152-44b8-995c-75bb6e1d6f16-kube-api-access-8l7kg\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:25:17.695085 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.694879 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkwjt\" (UniqueName: \"kubernetes.io/projected/47026a7c-8e2c-4c7e-ac69-4e027cc868aa-kube-api-access-xkwjt\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:25:17.789460 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:17.789419 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:25:18.420556 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:18.420521 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb" event={"ID":"a3c410c4-0152-44b8-995c-75bb6e1d6f16","Type":"ContainerDied","Data":"89098fde57c324a0ff2cb6c853959b2cc18d9cdbe5240c5f6f6d4483aa2c4651"} Aug 04 11:25:18.420974 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:18.420532 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb" Aug 04 11:25:18.420974 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:18.420612 2535 scope.go:117] "RemoveContainer" containerID="1f6fdd1450ce6fc3a73f713d535128434172024d6aa5b1c22e9ef9ace2aaa8c3" Aug 04 11:25:18.422004 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:18.421944 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m" Aug 04 11:25:18.422067 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:18.421946 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m" event={"ID":"47026a7c-8e2c-4c7e-ac69-4e027cc868aa","Type":"ContainerDied","Data":"41e61a48f503ef89b53e8b12e59b2045776cdbbf983164905df04db514e9294c"} Aug 04 11:25:18.430919 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:18.430903 2535 scope.go:117] "RemoveContainer" containerID="9d2ae7b661a9340c1dd86b18118b32aba1ef789b1b60d47ef33d4b29046a5c36" Aug 04 11:25:18.443064 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:18.443043 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb"] Aug 04 11:25:18.446898 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:18.446878 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-bc4a78-6d9864ccf8-n8kbb"] Aug 04 11:25:18.456745 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:18.456726 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m"] Aug 04 11:25:18.459080 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:18.459063 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-bc4a78-56b5485f88-kch5m"] Aug 04 11:25:18.480413 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:18.480386 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47026a7c-8e2c-4c7e-ac69-4e027cc868aa" path="/var/lib/kubelet/pods/47026a7c-8e2c-4c7e-ac69-4e027cc868aa/volumes" Aug 04 11:25:18.480789 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:18.480774 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f5708b8-549f-40f1-a6e3-44efd51e5dfb" path="/var/lib/kubelet/pods/7f5708b8-549f-40f1-a6e3-44efd51e5dfb/volumes" Aug 04 11:25:18.481125 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:18.481112 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3c410c4-0152-44b8-995c-75bb6e1d6f16" path="/var/lib/kubelet/pods/a3c410c4-0152-44b8-995c-75bb6e1d6f16/volumes" Aug 04 11:25:18.481384 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:18.481371 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4f86870-bc42-419a-902b-dba73521e0e9" path="/var/lib/kubelet/pods/b4f86870-bc42-419a-902b-dba73521e0e9/volumes" Aug 04 11:25:19.214690 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:19.214659 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:25:23.942153 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:23.942119 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:25:24.004052 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:24.004020 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:25:24.196461 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:24.196387 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:25:24.436407 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:24.436375 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:25:24.556875 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:24.556832 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-460cac-sp9qh"] Aug 04 11:25:24.557202 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:24.557188 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47026a7c-8e2c-4c7e-ac69-4e027cc868aa" containerName="payload-pre-processing" Aug 04 11:25:24.557202 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:24.557202 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="47026a7c-8e2c-4c7e-ac69-4e027cc868aa" containerName="payload-pre-processing" Aug 04 11:25:24.557351 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:24.557219 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3c410c4-0152-44b8-995c-75bb6e1d6f16" containerName="payload-processing" Aug 04 11:25:24.557351 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:24.557225 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3c410c4-0152-44b8-995c-75bb6e1d6f16" containerName="payload-processing" Aug 04 11:25:24.557351 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:24.557238 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f5708b8-549f-40f1-a6e3-44efd51e5dfb" containerName="revoke-keys" Aug 04 11:25:24.557351 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:24.557244 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f5708b8-549f-40f1-a6e3-44efd51e5dfb" containerName="revoke-keys" Aug 04 11:25:24.557351 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:24.557321 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3c410c4-0152-44b8-995c-75bb6e1d6f16" containerName="payload-processing" Aug 04 11:25:24.557351 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:24.557327 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="47026a7c-8e2c-4c7e-ac69-4e027cc868aa" containerName="payload-pre-processing" Aug 04 11:25:24.557351 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:24.557333 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f5708b8-549f-40f1-a6e3-44efd51e5dfb" containerName="revoke-keys" Aug 04 11:25:24.561833 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:24.561811 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-460cac-sp9qh" Aug 04 11:25:24.563783 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:24.563766 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qptbt\"" Aug 04 11:25:24.568477 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:24.568453 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-460cac-sp9qh"] Aug 04 11:25:24.656831 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:24.656801 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26229646-4778-4502-946f-b309574ffc9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-460cac-sp9qh\" (UID: \"26229646-4778-4502-946f-b309574ffc9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-460cac-sp9qh" Aug 04 11:25:24.757952 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:24.757921 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26229646-4778-4502-946f-b309574ffc9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-460cac-sp9qh\" (UID: \"26229646-4778-4502-946f-b309574ffc9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-460cac-sp9qh" Aug 04 11:25:24.758510 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:24.758471 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26229646-4778-4502-946f-b309574ffc9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-460cac-sp9qh\" (UID: \"26229646-4778-4502-946f-b309574ffc9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-460cac-sp9qh" Aug 04 11:25:24.873756 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:24.873682 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-460cac-sp9qh" Aug 04 11:25:25.011848 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:25.011813 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-460cac-sp9qh"] Aug 04 11:25:25.015117 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:25:25.015087 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26229646_4778_4502_946f_b309574ffc9f.slice/crio-d085138582374c5f1cdb3ad659c088e6af84c2facb8f7db8d91caec6c5dee16d WatchSource:0}: Error finding container d085138582374c5f1cdb3ad659c088e6af84c2facb8f7db8d91caec6c5dee16d: Status 404 returned error can't find the container with id d085138582374c5f1cdb3ad659c088e6af84c2facb8f7db8d91caec6c5dee16d Aug 04 11:25:25.459421 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:25.459332 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-460cac-sp9qh" event={"ID":"26229646-4778-4502-946f-b309574ffc9f","Type":"ContainerStarted","Data":"03fe9dd2bfac7c702c3deb5eedb8a9bb7f59ce094c36dd062ab81e1d2f7750b4"} Aug 04 11:25:25.459421 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:25.459397 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-460cac-sp9qh" event={"ID":"26229646-4778-4502-946f-b309574ffc9f","Type":"ContainerStarted","Data":"d085138582374c5f1cdb3ad659c088e6af84c2facb8f7db8d91caec6c5dee16d"} Aug 04 11:25:26.466668 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:26.466633 2535 generic.go:356] "Generic (PLEG): container finished" podID="26229646-4778-4502-946f-b309574ffc9f" containerID="03fe9dd2bfac7c702c3deb5eedb8a9bb7f59ce094c36dd062ab81e1d2f7750b4" exitCode=0 Aug 04 11:25:26.467091 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:26.466696 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-460cac-sp9qh" event={"ID":"26229646-4778-4502-946f-b309574ffc9f","Type":"ContainerDied","Data":"03fe9dd2bfac7c702c3deb5eedb8a9bb7f59ce094c36dd062ab81e1d2f7750b4"} Aug 04 11:25:27.598868 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:27.598845 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-460cac-sp9qh" Aug 04 11:25:27.686836 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:27.686802 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26229646-4778-4502-946f-b309574ffc9f-maas-api-service-ca\") pod \"26229646-4778-4502-946f-b309574ffc9f\" (UID: \"26229646-4778-4502-946f-b309574ffc9f\") " Aug 04 11:25:27.687160 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:27.687136 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26229646-4778-4502-946f-b309574ffc9f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "26229646-4778-4502-946f-b309574ffc9f" (UID: "26229646-4778-4502-946f-b309574ffc9f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:25:27.787831 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:27.787804 2535 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26229646-4778-4502-946f-b309574ffc9f-maas-api-service-ca\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:25:28.477736 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:28.477706 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-460cac-sp9qh" Aug 04 11:25:28.480885 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:28.480855 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-460cac-sp9qh" event={"ID":"26229646-4778-4502-946f-b309574ffc9f","Type":"ContainerDied","Data":"d085138582374c5f1cdb3ad659c088e6af84c2facb8f7db8d91caec6c5dee16d"} Aug 04 11:25:28.481015 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:28.480904 2535 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d085138582374c5f1cdb3ad659c088e6af84c2facb8f7db8d91caec6c5dee16d" Aug 04 11:25:34.993133 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:34.993095 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-460cac-29764035-fmbls"] Aug 04 11:25:34.995165 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:34.995137 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-460cac-29764035-fmbls"] Aug 04 11:25:34.998658 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:34.998623 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:25:35.022042 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.022010 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs"] Aug 04 11:25:35.022265 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.022236 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs" podUID="80d7c11a-a33a-473e-992f-da9d3f3f982a" containerName="payload-processing" containerID="cri-o://1be1834aaff81809128f68f87fa625128df368105ea5eceebcdf0672ca82bc0e" gracePeriod=30 Aug 04 11:25:35.031001 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.030972 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f"] Aug 04 11:25:35.031215 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.031193 2535 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f" podUID="da977de0-cee3-4fbb-b5ca-94c23f5884dc" containerName="payload-pre-processing" containerID="cri-o://0aa3664d2394917501cd7f473e580c76b274c06d2749676de6ac1c9657ae4343" gracePeriod=30 Aug 04 11:25:35.065119 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.065084 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:25:35.165771 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.161466 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-460cac-sp9qh"] Aug 04 11:25:35.165771 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.163474 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:25:35.168029 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.168001 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-460cac-sp9qh"] Aug 04 11:25:35.231944 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.231911 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:25:35.278100 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.278079 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs" Aug 04 11:25:35.315546 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.315520 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f" Aug 04 11:25:35.352681 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.352650 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80d7c11a-a33a-473e-992f-da9d3f3f982a-plugins-config-volume\") pod \"80d7c11a-a33a-473e-992f-da9d3f3f982a\" (UID: \"80d7c11a-a33a-473e-992f-da9d3f3f982a\") " Aug 04 11:25:35.352864 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.352754 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2jk4\" (UniqueName: \"kubernetes.io/projected/80d7c11a-a33a-473e-992f-da9d3f3f982a-kube-api-access-h2jk4\") pod \"80d7c11a-a33a-473e-992f-da9d3f3f982a\" (UID: \"80d7c11a-a33a-473e-992f-da9d3f3f982a\") " Aug 04 11:25:35.353022 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.353000 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80d7c11a-a33a-473e-992f-da9d3f3f982a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "80d7c11a-a33a-473e-992f-da9d3f3f982a" (UID: "80d7c11a-a33a-473e-992f-da9d3f3f982a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:25:35.354881 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.354855 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80d7c11a-a33a-473e-992f-da9d3f3f982a-kube-api-access-h2jk4" (OuterVolumeSpecName: "kube-api-access-h2jk4") pod "80d7c11a-a33a-473e-992f-da9d3f3f982a" (UID: "80d7c11a-a33a-473e-992f-da9d3f3f982a"). InnerVolumeSpecName "kube-api-access-h2jk4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 11:25:35.453653 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.453606 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xvnq9\" (UniqueName: \"kubernetes.io/projected/da977de0-cee3-4fbb-b5ca-94c23f5884dc-kube-api-access-xvnq9\") pod \"da977de0-cee3-4fbb-b5ca-94c23f5884dc\" (UID: \"da977de0-cee3-4fbb-b5ca-94c23f5884dc\") " Aug 04 11:25:35.453653 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.453669 2535 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da977de0-cee3-4fbb-b5ca-94c23f5884dc-plugins-config-volume\") pod \"da977de0-cee3-4fbb-b5ca-94c23f5884dc\" (UID: \"da977de0-cee3-4fbb-b5ca-94c23f5884dc\") " Aug 04 11:25:35.453924 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.453853 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2jk4\" (UniqueName: \"kubernetes.io/projected/80d7c11a-a33a-473e-992f-da9d3f3f982a-kube-api-access-h2jk4\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:25:35.453924 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.453864 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80d7c11a-a33a-473e-992f-da9d3f3f982a-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:25:35.454106 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.454081 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da977de0-cee3-4fbb-b5ca-94c23f5884dc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "da977de0-cee3-4fbb-b5ca-94c23f5884dc" (UID: "da977de0-cee3-4fbb-b5ca-94c23f5884dc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 11:25:35.455680 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.455658 2535 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da977de0-cee3-4fbb-b5ca-94c23f5884dc-kube-api-access-xvnq9" (OuterVolumeSpecName: "kube-api-access-xvnq9") pod "da977de0-cee3-4fbb-b5ca-94c23f5884dc" (UID: "da977de0-cee3-4fbb-b5ca-94c23f5884dc"). InnerVolumeSpecName "kube-api-access-xvnq9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 11:25:35.514619 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.514543 2535 generic.go:356] "Generic (PLEG): container finished" podID="80d7c11a-a33a-473e-992f-da9d3f3f982a" containerID="1be1834aaff81809128f68f87fa625128df368105ea5eceebcdf0672ca82bc0e" exitCode=0 Aug 04 11:25:35.514619 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.514579 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs" Aug 04 11:25:35.514835 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.514632 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs" event={"ID":"80d7c11a-a33a-473e-992f-da9d3f3f982a","Type":"ContainerDied","Data":"1be1834aaff81809128f68f87fa625128df368105ea5eceebcdf0672ca82bc0e"} Aug 04 11:25:35.514835 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.514689 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs" event={"ID":"80d7c11a-a33a-473e-992f-da9d3f3f982a","Type":"ContainerDied","Data":"1c13e1a45c7c934bc6bdd0ea493a967ac812806e31c014f1960837191f820b6b"} Aug 04 11:25:35.514835 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.514722 2535 scope.go:117] "RemoveContainer" containerID="1be1834aaff81809128f68f87fa625128df368105ea5eceebcdf0672ca82bc0e" Aug 04 11:25:35.515984 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.515965 2535 generic.go:356] "Generic (PLEG): container finished" podID="da977de0-cee3-4fbb-b5ca-94c23f5884dc" containerID="0aa3664d2394917501cd7f473e580c76b274c06d2749676de6ac1c9657ae4343" exitCode=0 Aug 04 11:25:35.516050 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.516002 2535 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f" Aug 04 11:25:35.516110 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.516049 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f" event={"ID":"da977de0-cee3-4fbb-b5ca-94c23f5884dc","Type":"ContainerDied","Data":"0aa3664d2394917501cd7f473e580c76b274c06d2749676de6ac1c9657ae4343"} Aug 04 11:25:35.516110 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.516100 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f" event={"ID":"da977de0-cee3-4fbb-b5ca-94c23f5884dc","Type":"ContainerDied","Data":"514306754cfc22437d17576bca49251f20efbfc17c351a74f4336b894771cf4d"} Aug 04 11:25:35.524376 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.524251 2535 scope.go:117] "RemoveContainer" containerID="1be1834aaff81809128f68f87fa625128df368105ea5eceebcdf0672ca82bc0e" Aug 04 11:25:35.524594 ip-10-0-130-250 kubenswrapper[2535]: E0804 11:25:35.524537 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1be1834aaff81809128f68f87fa625128df368105ea5eceebcdf0672ca82bc0e\": container with ID starting with 1be1834aaff81809128f68f87fa625128df368105ea5eceebcdf0672ca82bc0e not found: ID does not exist" containerID="1be1834aaff81809128f68f87fa625128df368105ea5eceebcdf0672ca82bc0e" Aug 04 11:25:35.524594 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.524560 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1be1834aaff81809128f68f87fa625128df368105ea5eceebcdf0672ca82bc0e"} err="failed to get container status \"1be1834aaff81809128f68f87fa625128df368105ea5eceebcdf0672ca82bc0e\": rpc error: code = NotFound desc = could not find container \"1be1834aaff81809128f68f87fa625128df368105ea5eceebcdf0672ca82bc0e\": container with ID starting with 1be1834aaff81809128f68f87fa625128df368105ea5eceebcdf0672ca82bc0e not found: ID does not exist" Aug 04 11:25:35.524594 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.524578 2535 scope.go:117] "RemoveContainer" containerID="0aa3664d2394917501cd7f473e580c76b274c06d2749676de6ac1c9657ae4343" Aug 04 11:25:35.532222 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.532207 2535 scope.go:117] "RemoveContainer" containerID="0aa3664d2394917501cd7f473e580c76b274c06d2749676de6ac1c9657ae4343" Aug 04 11:25:35.532452 ip-10-0-130-250 kubenswrapper[2535]: E0804 11:25:35.532435 2535 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0aa3664d2394917501cd7f473e580c76b274c06d2749676de6ac1c9657ae4343\": container with ID starting with 0aa3664d2394917501cd7f473e580c76b274c06d2749676de6ac1c9657ae4343 not found: ID does not exist" containerID="0aa3664d2394917501cd7f473e580c76b274c06d2749676de6ac1c9657ae4343" Aug 04 11:25:35.532515 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.532458 2535 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0aa3664d2394917501cd7f473e580c76b274c06d2749676de6ac1c9657ae4343"} err="failed to get container status \"0aa3664d2394917501cd7f473e580c76b274c06d2749676de6ac1c9657ae4343\": rpc error: code = NotFound desc = could not find container \"0aa3664d2394917501cd7f473e580c76b274c06d2749676de6ac1c9657ae4343\": container with ID starting with 0aa3664d2394917501cd7f473e580c76b274c06d2749676de6ac1c9657ae4343 not found: ID does not exist" Aug 04 11:25:35.538425 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.538402 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs"] Aug 04 11:25:35.542279 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.542259 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-460cac-569b6f78-wzpfs"] Aug 04 11:25:35.551586 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.551567 2535 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f"] Aug 04 11:25:35.554684 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.554665 2535 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvnq9\" (UniqueName: \"kubernetes.io/projected/da977de0-cee3-4fbb-b5ca-94c23f5884dc-kube-api-access-xvnq9\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:25:35.554766 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.554688 2535 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da977de0-cee3-4fbb-b5ca-94c23f5884dc-plugins-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Aug 04 11:25:35.555324 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.555304 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:25:35.555746 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:35.555729 2535 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-460cac-f9478f8c6-t5r8f"] Aug 04 11:25:36.481143 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:36.481062 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26229646-4778-4502-946f-b309574ffc9f" path="/var/lib/kubelet/pods/26229646-4778-4502-946f-b309574ffc9f/volumes" Aug 04 11:25:36.481583 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:36.481566 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80d7c11a-a33a-473e-992f-da9d3f3f982a" path="/var/lib/kubelet/pods/80d7c11a-a33a-473e-992f-da9d3f3f982a/volumes" Aug 04 11:25:36.481918 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:36.481905 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d88a2c93-a89f-4001-8f90-7fb8ef86bf60" path="/var/lib/kubelet/pods/d88a2c93-a89f-4001-8f90-7fb8ef86bf60/volumes" Aug 04 11:25:36.482288 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:36.482274 2535 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da977de0-cee3-4fbb-b5ca-94c23f5884dc" path="/var/lib/kubelet/pods/da977de0-cee3-4fbb-b5ca-94c23f5884dc/volumes" Aug 04 11:25:41.771356 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:41.771314 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:25:41.871996 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:41.871959 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:25:42.056930 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:42.056899 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:25:42.336239 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:42.336158 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:25:42.875856 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:42.875827 2535 scope.go:117] "RemoveContainer" containerID="374e690473ac8b4ba351bc5ae08ca18c1d6c7352917b812ae605e8ab5f18030a" Aug 04 11:25:42.884013 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:42.883991 2535 scope.go:117] "RemoveContainer" containerID="cf8a7694901996a3664b41fef6a48e126d70d3089179c8be3c7e94fd55d65155" Aug 04 11:25:48.507057 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:48.507019 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-67c8859cd8-25fr9_70003f72-bdcc-40b1-bf11-a7c546adc577/manager/0.log" Aug 04 11:25:49.126662 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:49.126635 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-598d67d848-dr22z_492e0c27-9f41-4a7c-b4ca-f93d3bd05d86/copy-manifests/0.log" Aug 04 11:25:49.140183 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:49.140158 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-598d67d848-dr22z_492e0c27-9f41-4a7c-b4ca-f93d3bd05d86/manager/0.log" Aug 04 11:25:49.616181 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:49.616153 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29764005-dpv9v_fc4a02ed-caf3-4f3c-9235-b0fe6db253da/cleanup/0.log" Aug 04 11:25:49.729397 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:49.729370 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29764020-28nbf_ac1f65a7-7177-4828-8a79-5e79e204c3a5/cleanup/0.log" Aug 04 11:25:49.837996 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:49.837971 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29764035-lc5g4_cac02ebe-61ae-4524-8ae1-4473d1596545/cleanup/0.log" Aug 04 11:25:49.952981 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:49.952912 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-hzmvq_bc1e76a3-d78b-4d45-8140-891118fe0a51/postgres/0.log" Aug 04 11:25:51.944223 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:51.944193 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-6wrhv_a1a1cd50-d008-4bf4-89bb-43afe394c465/discovery/0.log" Aug 04 11:25:52.414375 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:52.414343 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-zxvgh_c71d8203-4632-4601-a247-8d255666d0d5/payload-pre-processing/0.log" Aug 04 11:25:52.530893 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:52.530856 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-whd9j_e2d7c7cf-b757-4023-a277-38570756f493/payload-processing/0.log" Aug 04 11:25:52.987355 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:52.987327 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-pl2rw_776675a2-a82b-454d-a0d1-ea8140d2789b/main/0.log" Aug 04 11:25:53.100987 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:53.100957 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-mq8lv_f66196c5-ae6e-4922-aa61-61436f504b4e/main/0.log" Aug 04 11:25:53.333730 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:53.333703 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fw57k2_54995a67-600f-4bda-bbd0-c86307e0df08/main/0.log" Aug 04 11:25:59.342794 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:59.342760 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:25:59.407142 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:25:59.407110 2535 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" limit=1024 Aug 04 11:26:02.277737 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:02.277705 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zzlbv_73272ce5-1238-42c7-b697-29d21e373fc0/global-pull-secret-syncer/0.log" Aug 04 11:26:02.325593 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:02.325545 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5bmt4_ad9f7eb2-071c-4803-96f1-649155c6f4db/konnectivity-agent/0.log" Aug 04 11:26:02.439379 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:02.439352 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-250.ec2.internal_b7c39b691124fb3a351388d810da798f/haproxy/0.log" Aug 04 11:26:10.061311 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:10.061285 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-9d8mh_3c0c22b1-8555-4ff4-9ead-afe3b4a58817/kube-state-metrics/0.log" Aug 04 11:26:10.078974 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:10.078942 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-9d8mh_3c0c22b1-8555-4ff4-9ead-afe3b4a58817/kube-rbac-proxy-main/0.log" Aug 04 11:26:10.100674 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:10.100653 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-9d8mh_3c0c22b1-8555-4ff4-9ead-afe3b4a58817/kube-rbac-proxy-self/0.log" Aug 04 11:26:10.388454 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:10.388430 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w798t_4fda4dea-ac20-4e18-8861-80a87970ca59/node-exporter/0.log" Aug 04 11:26:10.421811 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:10.421786 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w798t_4fda4dea-ac20-4e18-8861-80a87970ca59/kube-rbac-proxy/0.log" Aug 04 11:26:10.445636 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:10.445613 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w798t_4fda4dea-ac20-4e18-8861-80a87970ca59/init-textfile/0.log" Aug 04 11:26:10.569737 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:10.569705 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2be47b9e-0e11-4bc8-9608-158357a2f2d2/prometheus/0.log" Aug 04 11:26:10.589171 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:10.589112 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2be47b9e-0e11-4bc8-9608-158357a2f2d2/config-reloader/0.log" Aug 04 11:26:10.608999 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:10.608974 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2be47b9e-0e11-4bc8-9608-158357a2f2d2/thanos-sidecar/0.log" Aug 04 11:26:10.630997 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:10.630981 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2be47b9e-0e11-4bc8-9608-158357a2f2d2/kube-rbac-proxy-web/0.log" Aug 04 11:26:10.664249 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:10.664227 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2be47b9e-0e11-4bc8-9608-158357a2f2d2/kube-rbac-proxy/0.log" Aug 04 11:26:10.690946 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:10.690925 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2be47b9e-0e11-4bc8-9608-158357a2f2d2/kube-rbac-proxy-thanos/0.log" Aug 04 11:26:10.712611 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:10.712585 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2be47b9e-0e11-4bc8-9608-158357a2f2d2/init-config-reloader/0.log" Aug 04 11:26:10.753307 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:10.753279 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-758d7cb874-xlkqn_c6ade897-5b7d-49bf-a4ef-e01062cadf09/prometheus-operator/0.log" Aug 04 11:26:10.770715 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:10.770693 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-758d7cb874-xlkqn_c6ade897-5b7d-49bf-a4ef-e01062cadf09/kube-rbac-proxy/0.log" Aug 04 11:26:10.796076 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:10.796053 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-66685ff6d4-vrblv_0c82316b-a816-4c9e-b307-686d0e17e679/prometheus-operator-admission-webhook/0.log" Aug 04 11:26:12.678464 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.678430 2535 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qfz6h/perf-node-gather-daemonset-smrdm"] Aug 04 11:26:12.678912 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.678805 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da977de0-cee3-4fbb-b5ca-94c23f5884dc" containerName="payload-pre-processing" Aug 04 11:26:12.678912 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.678816 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="da977de0-cee3-4fbb-b5ca-94c23f5884dc" containerName="payload-pre-processing" Aug 04 11:26:12.678912 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.678825 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26229646-4778-4502-946f-b309574ffc9f" containerName="revoke-keys" Aug 04 11:26:12.678912 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.678831 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="26229646-4778-4502-946f-b309574ffc9f" containerName="revoke-keys" Aug 04 11:26:12.678912 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.678839 2535 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80d7c11a-a33a-473e-992f-da9d3f3f982a" containerName="payload-processing" Aug 04 11:26:12.678912 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.678846 2535 state_mem.go:107] "Deleted CPUSet assignment" podUID="80d7c11a-a33a-473e-992f-da9d3f3f982a" containerName="payload-processing" Aug 04 11:26:12.678912 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.678908 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="26229646-4778-4502-946f-b309574ffc9f" containerName="revoke-keys" Aug 04 11:26:12.679130 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.678921 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="da977de0-cee3-4fbb-b5ca-94c23f5884dc" containerName="payload-pre-processing" Aug 04 11:26:12.679130 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.678928 2535 memory_manager.go:368] "RemoveStaleState removing state" podUID="80d7c11a-a33a-473e-992f-da9d3f3f982a" containerName="payload-processing" Aug 04 11:26:12.681316 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.681296 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qfz6h/perf-node-gather-daemonset-smrdm" Aug 04 11:26:12.683245 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.683225 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qfz6h\"/\"openshift-service-ca.crt\"" Aug 04 11:26:12.683363 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.683343 2535 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qfz6h\"/\"kube-root-ca.crt\"" Aug 04 11:26:12.683697 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.683682 2535 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qfz6h\"/\"default-dockercfg-67krk\"" Aug 04 11:26:12.690775 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.690752 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qfz6h/perf-node-gather-daemonset-smrdm"] Aug 04 11:26:12.778180 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.778149 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2a683414-b43e-4f91-bd32-963259b3e784-podres\") pod \"perf-node-gather-daemonset-smrdm\" (UID: \"2a683414-b43e-4f91-bd32-963259b3e784\") " pod="openshift-must-gather-qfz6h/perf-node-gather-daemonset-smrdm" Aug 04 11:26:12.778356 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.778196 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2a683414-b43e-4f91-bd32-963259b3e784-proc\") pod \"perf-node-gather-daemonset-smrdm\" (UID: \"2a683414-b43e-4f91-bd32-963259b3e784\") " pod="openshift-must-gather-qfz6h/perf-node-gather-daemonset-smrdm" Aug 04 11:26:12.778356 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.778212 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt6nc\" (UniqueName: \"kubernetes.io/projected/2a683414-b43e-4f91-bd32-963259b3e784-kube-api-access-gt6nc\") pod \"perf-node-gather-daemonset-smrdm\" (UID: \"2a683414-b43e-4f91-bd32-963259b3e784\") " pod="openshift-must-gather-qfz6h/perf-node-gather-daemonset-smrdm" Aug 04 11:26:12.778356 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.778275 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2a683414-b43e-4f91-bd32-963259b3e784-sys\") pod \"perf-node-gather-daemonset-smrdm\" (UID: \"2a683414-b43e-4f91-bd32-963259b3e784\") " pod="openshift-must-gather-qfz6h/perf-node-gather-daemonset-smrdm" Aug 04 11:26:12.778356 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.778329 2535 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2a683414-b43e-4f91-bd32-963259b3e784-lib-modules\") pod \"perf-node-gather-daemonset-smrdm\" (UID: \"2a683414-b43e-4f91-bd32-963259b3e784\") " pod="openshift-must-gather-qfz6h/perf-node-gather-daemonset-smrdm" Aug 04 11:26:12.879533 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.879504 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2a683414-b43e-4f91-bd32-963259b3e784-proc\") pod \"perf-node-gather-daemonset-smrdm\" (UID: \"2a683414-b43e-4f91-bd32-963259b3e784\") " pod="openshift-must-gather-qfz6h/perf-node-gather-daemonset-smrdm" Aug 04 11:26:12.879689 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.879539 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt6nc\" (UniqueName: \"kubernetes.io/projected/2a683414-b43e-4f91-bd32-963259b3e784-kube-api-access-gt6nc\") pod \"perf-node-gather-daemonset-smrdm\" (UID: \"2a683414-b43e-4f91-bd32-963259b3e784\") " pod="openshift-must-gather-qfz6h/perf-node-gather-daemonset-smrdm" Aug 04 11:26:12.879689 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.879558 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2a683414-b43e-4f91-bd32-963259b3e784-proc\") pod \"perf-node-gather-daemonset-smrdm\" (UID: \"2a683414-b43e-4f91-bd32-963259b3e784\") " pod="openshift-must-gather-qfz6h/perf-node-gather-daemonset-smrdm" Aug 04 11:26:12.879689 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.879561 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2a683414-b43e-4f91-bd32-963259b3e784-sys\") pod \"perf-node-gather-daemonset-smrdm\" (UID: \"2a683414-b43e-4f91-bd32-963259b3e784\") " pod="openshift-must-gather-qfz6h/perf-node-gather-daemonset-smrdm" Aug 04 11:26:12.879689 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.879604 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2a683414-b43e-4f91-bd32-963259b3e784-sys\") pod \"perf-node-gather-daemonset-smrdm\" (UID: \"2a683414-b43e-4f91-bd32-963259b3e784\") " pod="openshift-must-gather-qfz6h/perf-node-gather-daemonset-smrdm" Aug 04 11:26:12.879689 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.879648 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2a683414-b43e-4f91-bd32-963259b3e784-lib-modules\") pod \"perf-node-gather-daemonset-smrdm\" (UID: \"2a683414-b43e-4f91-bd32-963259b3e784\") " pod="openshift-must-gather-qfz6h/perf-node-gather-daemonset-smrdm" Aug 04 11:26:12.879880 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.879695 2535 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2a683414-b43e-4f91-bd32-963259b3e784-podres\") pod \"perf-node-gather-daemonset-smrdm\" (UID: \"2a683414-b43e-4f91-bd32-963259b3e784\") " pod="openshift-must-gather-qfz6h/perf-node-gather-daemonset-smrdm" Aug 04 11:26:12.879880 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.879793 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2a683414-b43e-4f91-bd32-963259b3e784-podres\") pod \"perf-node-gather-daemonset-smrdm\" (UID: \"2a683414-b43e-4f91-bd32-963259b3e784\") " pod="openshift-must-gather-qfz6h/perf-node-gather-daemonset-smrdm" Aug 04 11:26:12.879880 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.879797 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2a683414-b43e-4f91-bd32-963259b3e784-lib-modules\") pod \"perf-node-gather-daemonset-smrdm\" (UID: \"2a683414-b43e-4f91-bd32-963259b3e784\") " pod="openshift-must-gather-qfz6h/perf-node-gather-daemonset-smrdm" Aug 04 11:26:12.887757 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.887732 2535 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt6nc\" (UniqueName: \"kubernetes.io/projected/2a683414-b43e-4f91-bd32-963259b3e784-kube-api-access-gt6nc\") pod \"perf-node-gather-daemonset-smrdm\" (UID: \"2a683414-b43e-4f91-bd32-963259b3e784\") " pod="openshift-must-gather-qfz6h/perf-node-gather-daemonset-smrdm" Aug 04 11:26:12.991830 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:12.991767 2535 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qfz6h/perf-node-gather-daemonset-smrdm" Aug 04 11:26:13.117258 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:13.117233 2535 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qfz6h/perf-node-gather-daemonset-smrdm"] Aug 04 11:26:13.119450 ip-10-0-130-250 kubenswrapper[2535]: W0804 11:26:13.119410 2535 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2a683414_b43e_4f91_bd32_963259b3e784.slice/crio-674d2af9a635b36b020531fbc2098f30cabca4919589ef665a500449a34733fb WatchSource:0}: Error finding container 674d2af9a635b36b020531fbc2098f30cabca4919589ef665a500449a34733fb: Status 404 returned error can't find the container with id 674d2af9a635b36b020531fbc2098f30cabca4919589ef665a500449a34733fb Aug 04 11:26:13.457912 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:13.457879 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-77c974ffcf-s6xxv_45b2091a-93db-4ee5-b0b7-116c9ae5ef72/download-server/0.log" Aug 04 11:26:13.715521 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:13.715419 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qfz6h/perf-node-gather-daemonset-smrdm" event={"ID":"2a683414-b43e-4f91-bd32-963259b3e784","Type":"ContainerStarted","Data":"130bf4768a28b8dbda32dadb05c12819b1e977c58dc72bf96ed9fac9094ce1f7"} Aug 04 11:26:13.715521 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:13.715480 2535 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qfz6h/perf-node-gather-daemonset-smrdm" event={"ID":"2a683414-b43e-4f91-bd32-963259b3e784","Type":"ContainerStarted","Data":"674d2af9a635b36b020531fbc2098f30cabca4919589ef665a500449a34733fb"} Aug 04 11:26:13.715904 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:13.715526 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qfz6h/perf-node-gather-daemonset-smrdm" Aug 04 11:26:13.734357 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:13.734308 2535 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qfz6h/perf-node-gather-daemonset-smrdm" podStartSLOduration=1.734291733 podStartE2EDuration="1.734291733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 11:26:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 11:26:13.732093379 +0000 UTC m=+3511.706175937" watchObservedRunningTime="2026-08-04 11:26:13.734291733 +0000 UTC m=+3511.708374259" Aug 04 11:26:15.093193 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:15.093157 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7wdtv_5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99/dns/0.log" Aug 04 11:26:15.116480 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:15.116457 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7wdtv_5fcf23dc-6a51-44e4-b4cc-dd2a77fe1c99/kube-rbac-proxy/0.log" Aug 04 11:26:15.275952 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:15.275918 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s629f_696380db-a93f-48d7-9bb7-108052391f48/dns-node-resolver/0.log" Aug 04 11:26:15.791589 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:15.791557 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-74d45d67dc-rb8sl_7a214a2a-3267-48bb-8cd4-03f8fcf5bce6/registry/0.log" Aug 04 11:26:15.864024 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:15.863986 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zpdbz_9be224ce-6218-4f85-a9f0-a09bf05d4f25/node-ca/0.log" Aug 04 11:26:16.945839 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:16.945804 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-6wrhv_a1a1cd50-d008-4bf4-89bb-43afe394c465/discovery/0.log" Aug 04 11:26:17.060745 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:17.060717 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-zxvgh_c71d8203-4632-4601-a247-8d255666d0d5/payload-pre-processing/0.log" Aug 04 11:26:17.088827 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:17.088799 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-whd9j_e2d7c7cf-b757-4023-a277-38570756f493/payload-processing/0.log" Aug 04 11:26:17.951196 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:17.951161 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ttzdq_1955136f-4af9-4ab3-81df-790fa97cddab/serve-healthcheck-canary/0.log" Aug 04 11:26:18.560504 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:18.560456 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tgpbn_51786990-2492-4142-994e-9e908bd478c1/kube-rbac-proxy/0.log" Aug 04 11:26:18.577543 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:18.577518 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tgpbn_51786990-2492-4142-994e-9e908bd478c1/exporter/0.log" Aug 04 11:26:18.594837 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:18.594818 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tgpbn_51786990-2492-4142-994e-9e908bd478c1/extractor/0.log" Aug 04 11:26:19.729999 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:19.729968 2535 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qfz6h/perf-node-gather-daemonset-smrdm" Aug 04 11:26:21.233826 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:21.233782 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-67c8859cd8-25fr9_70003f72-bdcc-40b1-bf11-a7c546adc577/manager/0.log" Aug 04 11:26:21.547318 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:21.547289 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-598d67d848-dr22z_492e0c27-9f41-4a7c-b4ca-f93d3bd05d86/manager/0.log" Aug 04 11:26:21.566762 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:21.566741 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-598d67d848-dr22z_492e0c27-9f41-4a7c-b4ca-f93d3bd05d86/copy-manifests/0.log" Aug 04 11:26:22.987376 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:22.987350 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-59766cd755-pl4cj_b047bdce-f1e4-4ffe-92d3-ffaf4d225065/manager/0.log" Aug 04 11:26:23.029824 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:23.029799 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-jqp9s_abaa863c-5c3f-44c8-aab6-c04da3416463/openshift-lws-operator/0.log" Aug 04 11:26:23.443542 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:23.443514 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-6wrhv_a1a1cd50-d008-4bf4-89bb-43afe394c465/discovery/0.log" Aug 04 11:26:23.514193 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:23.514163 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-zxvgh_c71d8203-4632-4601-a247-8d255666d0d5/payload-pre-processing/0.log" Aug 04 11:26:23.531687 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:23.531658 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-whd9j_e2d7c7cf-b757-4023-a277-38570756f493/payload-processing/0.log" Aug 04 11:26:25.091235 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:25.091204 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-6wrhv_a1a1cd50-d008-4bf4-89bb-43afe394c465/discovery/0.log" Aug 04 11:26:25.178889 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:25.178858 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-zxvgh_c71d8203-4632-4601-a247-8d255666d0d5/payload-pre-processing/0.log" Aug 04 11:26:25.201358 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:25.201335 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-whd9j_e2d7c7cf-b757-4023-a277-38570756f493/payload-processing/0.log" Aug 04 11:26:28.031666 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:28.031633 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-68bc745d87-6cskx_da80e736-cae4-4cde-a20e-e54d05afe4b7/migrator/0.log" Aug 04 11:26:28.048503 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:28.048467 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-68bc745d87-6cskx_da80e736-cae4-4cde-a20e-e54d05afe4b7/graceful-termination/0.log" Aug 04 11:26:29.533620 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:29.533586 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-6xqqk_c581a5bc-9508-4bc7-bc61-6bbcc0bf0af9/kube-multus/0.log" Aug 04 11:26:29.886314 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:29.886236 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s59fm_7298c7a0-dc2b-4480-aa51-85ba014dcec6/kube-multus-additional-cni-plugins/0.log" Aug 04 11:26:29.909790 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:29.909765 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s59fm_7298c7a0-dc2b-4480-aa51-85ba014dcec6/egress-router-binary-copy/0.log" Aug 04 11:26:29.928138 ip-10-0-130-250 kubenswrapper[2535]: I0804 11:26:29.928113 2535 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s59fm_7298c7a0-dc2b-4480-aa51-85ba014dcec6/cni-plugins/0.log"