Aug 04 09:40:29.116612 ip-10-0-129-241 systemd[1]: Starting Kubernetes Kubelet... Aug 04 09:40:31.525563 ip-10-0-129-241 kubenswrapper[2572]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 09:40:31.525563 ip-10-0-129-241 kubenswrapper[2572]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Aug 04 09:40:31.525563 ip-10-0-129-241 kubenswrapper[2572]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 09:40:31.525563 ip-10-0-129-241 kubenswrapper[2572]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Aug 04 09:40:31.525563 ip-10-0-129-241 kubenswrapper[2572]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 09:40:31.526611 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.526464 2572 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Aug 04 09:40:31.530145 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530127 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 09:40:31.530145 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530145 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 09:40:31.530212 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530149 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 09:40:31.530212 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530152 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 09:40:31.530212 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530156 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 09:40:31.530212 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530160 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 09:40:31.530212 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530163 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 09:40:31.530212 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530166 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 09:40:31.530212 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530169 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 09:40:31.530212 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530172 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 09:40:31.530212 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530175 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 09:40:31.530212 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530178 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 09:40:31.530212 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530181 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 09:40:31.530212 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530184 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 09:40:31.530212 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530186 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 09:40:31.530212 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530189 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 09:40:31.530212 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530192 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 09:40:31.530212 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530195 2572 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 09:40:31.530212 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530198 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 09:40:31.530212 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530200 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 09:40:31.530212 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530203 2572 feature_gate.go:328] unrecognized feature gate: Example Aug 04 09:40:31.530212 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530205 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 09:40:31.530742 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530207 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 09:40:31.530742 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530219 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 09:40:31.530742 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530222 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 09:40:31.530742 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530225 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 09:40:31.530742 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530228 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 09:40:31.530742 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530231 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 09:40:31.530742 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530233 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 09:40:31.530742 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530236 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 09:40:31.530742 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530239 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 09:40:31.530742 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530241 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 09:40:31.530742 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530243 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 09:40:31.530742 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530249 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 09:40:31.530742 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530253 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 09:40:31.530742 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530256 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 09:40:31.530742 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530258 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 09:40:31.530742 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530261 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 09:40:31.530742 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530264 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 09:40:31.530742 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530266 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 09:40:31.530742 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530269 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 09:40:31.531229 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530272 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 09:40:31.531229 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530275 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 09:40:31.531229 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530277 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 09:40:31.531229 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530280 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 09:40:31.531229 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530282 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 09:40:31.531229 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530286 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 09:40:31.531229 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530289 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 09:40:31.531229 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530292 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 09:40:31.531229 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530294 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 09:40:31.531229 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530297 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 09:40:31.531229 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530299 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 09:40:31.531229 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530302 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 09:40:31.531229 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530304 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 09:40:31.531229 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530306 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 09:40:31.531229 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530310 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 09:40:31.531229 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530313 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 09:40:31.531229 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530316 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 09:40:31.531229 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530318 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 09:40:31.531229 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530321 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 09:40:31.531229 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530323 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 09:40:31.531719 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530327 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 09:40:31.531719 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530329 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 09:40:31.531719 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530332 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 09:40:31.531719 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530335 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 09:40:31.531719 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530337 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 09:40:31.531719 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530340 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 09:40:31.531719 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530342 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 09:40:31.531719 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530344 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 09:40:31.531719 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530347 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 09:40:31.531719 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530349 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 09:40:31.531719 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530352 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 09:40:31.531719 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530354 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 09:40:31.531719 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530357 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 09:40:31.531719 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530359 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 09:40:31.531719 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530363 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 09:40:31.531719 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530365 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 09:40:31.531719 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530368 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 09:40:31.531719 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530370 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 09:40:31.531719 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530373 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 09:40:31.532206 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530376 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 09:40:31.532206 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530378 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 09:40:31.532206 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530381 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 09:40:31.532206 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530383 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 09:40:31.532206 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530386 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 09:40:31.532206 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530388 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 09:40:31.532206 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530391 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 09:40:31.532206 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530396 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 09:40:31.532206 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530399 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 09:40:31.532206 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530402 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 09:40:31.532206 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530405 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 09:40:31.532206 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530407 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 09:40:31.532206 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530411 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 09:40:31.532206 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530414 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 09:40:31.532206 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530417 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 09:40:31.532206 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530420 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 09:40:31.532206 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530422 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 09:40:31.532206 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530425 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 09:40:31.532206 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.530427 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 09:40:31.532670 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531151 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 09:40:31.532670 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531157 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 09:40:31.532670 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531161 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 09:40:31.532670 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531164 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 09:40:31.532670 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531166 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 09:40:31.532670 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531169 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 09:40:31.532670 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531172 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 09:40:31.532670 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531175 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 09:40:31.532670 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531177 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 09:40:31.532670 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531179 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 09:40:31.532670 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531182 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 09:40:31.532670 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531185 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 09:40:31.532670 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531187 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 09:40:31.532670 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531190 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 09:40:31.532670 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531194 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 09:40:31.532670 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531197 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 09:40:31.532670 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531199 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 09:40:31.532670 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531202 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 09:40:31.532670 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531204 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 09:40:31.532670 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531207 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 09:40:31.533182 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531210 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 09:40:31.533182 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531213 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 09:40:31.533182 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531216 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 09:40:31.533182 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531219 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 09:40:31.533182 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531221 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 09:40:31.533182 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531224 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 09:40:31.533182 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531226 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 09:40:31.533182 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531229 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 09:40:31.533182 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531232 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 09:40:31.533182 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531234 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 09:40:31.533182 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531237 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 09:40:31.533182 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531240 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 09:40:31.533182 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531242 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 09:40:31.533182 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531245 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 09:40:31.533182 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531248 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 09:40:31.533182 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531250 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 09:40:31.533182 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531253 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 09:40:31.533182 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531256 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 09:40:31.533182 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531260 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 09:40:31.533182 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531264 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 09:40:31.533659 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531268 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 09:40:31.533659 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531272 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 09:40:31.533659 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531276 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 09:40:31.533659 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531279 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 09:40:31.533659 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531282 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 09:40:31.533659 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531285 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 09:40:31.533659 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531287 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 09:40:31.533659 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531290 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 09:40:31.533659 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531293 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 09:40:31.533659 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531296 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 09:40:31.533659 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531298 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 09:40:31.533659 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531300 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 09:40:31.533659 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531303 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 09:40:31.533659 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531306 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 09:40:31.533659 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531309 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 09:40:31.533659 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531311 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 09:40:31.533659 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531314 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 09:40:31.533659 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531318 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 09:40:31.533659 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531321 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 09:40:31.534155 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531324 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 09:40:31.534155 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531327 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 09:40:31.534155 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531330 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 09:40:31.534155 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531333 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 09:40:31.534155 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531335 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 09:40:31.534155 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531338 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 09:40:31.534155 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531340 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 09:40:31.534155 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531343 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 09:40:31.534155 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531346 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 09:40:31.534155 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531348 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 09:40:31.534155 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531351 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 09:40:31.534155 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531353 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 09:40:31.534155 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531356 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 09:40:31.534155 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531358 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 09:40:31.534155 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531361 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 09:40:31.534155 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531363 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 09:40:31.534155 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531367 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 09:40:31.534155 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531369 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 09:40:31.534155 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531372 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 09:40:31.534626 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531374 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 09:40:31.534626 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531377 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 09:40:31.534626 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531379 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 09:40:31.534626 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531382 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 09:40:31.534626 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531384 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 09:40:31.534626 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531387 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 09:40:31.534626 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531391 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 09:40:31.534626 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531394 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 09:40:31.534626 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531396 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 09:40:31.534626 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531399 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 09:40:31.534626 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531401 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 09:40:31.534626 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531404 2572 feature_gate.go:328] unrecognized feature gate: Example Aug 04 09:40:31.534626 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531406 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 09:40:31.534626 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531409 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 09:40:31.534626 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531411 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 09:40:31.534626 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531414 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 09:40:31.534626 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531416 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 09:40:31.534626 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531419 2572 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 09:40:31.534626 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531422 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 09:40:31.534626 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531424 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 09:40:31.535135 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531427 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 09:40:31.535135 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531863 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 09:40:31.535135 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531869 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 09:40:31.535135 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531872 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 09:40:31.535135 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531876 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 09:40:31.535135 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531878 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 09:40:31.535135 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531881 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 09:40:31.535135 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531884 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 09:40:31.535135 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531887 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 09:40:31.535135 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531890 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 09:40:31.535135 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531892 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 09:40:31.535135 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531895 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 09:40:31.535135 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531897 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 09:40:31.535135 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531900 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 09:40:31.535135 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531903 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 09:40:31.535135 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531906 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 09:40:31.535135 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531909 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 09:40:31.535135 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531911 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 09:40:31.535135 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531917 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 09:40:31.535609 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531921 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 09:40:31.535609 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531924 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 09:40:31.535609 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531927 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 09:40:31.535609 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531930 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 09:40:31.535609 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531933 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 09:40:31.535609 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531936 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 09:40:31.535609 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531939 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 09:40:31.535609 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531942 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 09:40:31.535609 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531944 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 09:40:31.535609 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531947 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 09:40:31.535609 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531949 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 09:40:31.535609 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531952 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 09:40:31.535609 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531954 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 09:40:31.535609 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531957 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 09:40:31.535609 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531960 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 09:40:31.535609 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531962 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 09:40:31.535609 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531965 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 09:40:31.535609 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531969 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 09:40:31.535609 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531972 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 09:40:31.536112 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531975 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 09:40:31.536112 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531977 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 09:40:31.536112 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531980 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 09:40:31.536112 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531983 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 09:40:31.536112 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531985 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 09:40:31.536112 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531988 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 09:40:31.536112 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531990 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 09:40:31.536112 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531993 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 09:40:31.536112 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531995 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 09:40:31.536112 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.531999 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 09:40:31.536112 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532003 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 09:40:31.536112 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532006 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 09:40:31.536112 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532010 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 09:40:31.536112 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532013 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 09:40:31.536112 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532016 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 09:40:31.536112 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532019 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 09:40:31.536112 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532022 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 09:40:31.536112 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532025 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 09:40:31.536112 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532028 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 09:40:31.536112 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532030 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 09:40:31.536582 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532033 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 09:40:31.536582 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532035 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 09:40:31.536582 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532038 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 09:40:31.536582 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532041 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 09:40:31.536582 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532043 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 09:40:31.536582 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532046 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 09:40:31.536582 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532048 2572 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 09:40:31.536582 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532051 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 09:40:31.536582 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532054 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 09:40:31.536582 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532056 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 09:40:31.536582 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532059 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 09:40:31.536582 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532061 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 09:40:31.536582 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532064 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 09:40:31.536582 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532066 2572 feature_gate.go:328] unrecognized feature gate: Example Aug 04 09:40:31.536582 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532069 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 09:40:31.536582 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532072 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 09:40:31.536582 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532074 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 09:40:31.536582 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532077 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 09:40:31.536582 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532080 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 09:40:31.536582 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532082 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 09:40:31.537078 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532085 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 09:40:31.537078 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532087 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 09:40:31.537078 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532090 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 09:40:31.537078 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532095 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 09:40:31.537078 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532098 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 09:40:31.537078 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532101 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 09:40:31.537078 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532103 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 09:40:31.537078 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532106 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 09:40:31.537078 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532108 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 09:40:31.537078 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532111 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 09:40:31.537078 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532113 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 09:40:31.537078 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532116 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 09:40:31.537078 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532119 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 09:40:31.537078 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532121 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 09:40:31.537078 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532124 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 09:40:31.537078 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532126 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 09:40:31.537078 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532129 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 09:40:31.537078 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532131 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 09:40:31.537078 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532134 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 09:40:31.537078 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532136 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 09:40:31.537603 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532139 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 09:40:31.537603 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532142 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 09:40:31.537603 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532211 2572 flags.go:64] FLAG: --address="0.0.0.0" Aug 04 09:40:31.537603 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532219 2572 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Aug 04 09:40:31.537603 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532225 2572 flags.go:64] FLAG: --anonymous-auth="true" Aug 04 09:40:31.537603 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532230 2572 flags.go:64] FLAG: --application-metrics-count-limit="100" Aug 04 09:40:31.537603 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532234 2572 flags.go:64] FLAG: --authentication-token-webhook="false" Aug 04 09:40:31.537603 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532237 2572 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Aug 04 09:40:31.537603 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532241 2572 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Aug 04 09:40:31.537603 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532245 2572 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Aug 04 09:40:31.537603 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532249 2572 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Aug 04 09:40:31.537603 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532252 2572 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Aug 04 09:40:31.537603 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532255 2572 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Aug 04 09:40:31.537603 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532261 2572 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Aug 04 09:40:31.537603 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532264 2572 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Aug 04 09:40:31.537603 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532268 2572 flags.go:64] FLAG: --cgroup-root="" Aug 04 09:40:31.537603 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532272 2572 flags.go:64] FLAG: --cgroups-per-qos="true" Aug 04 09:40:31.537603 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532275 2572 flags.go:64] FLAG: --client-ca-file="" Aug 04 09:40:31.537603 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532278 2572 flags.go:64] FLAG: --cloud-provider="external" Aug 04 09:40:31.537603 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532281 2572 flags.go:64] FLAG: --cluster-dns="[]" Aug 04 09:40:31.537603 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532287 2572 flags.go:64] FLAG: --cluster-domain="" Aug 04 09:40:31.537603 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532289 2572 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Aug 04 09:40:31.537603 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532293 2572 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Aug 04 09:40:31.537603 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532296 2572 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Aug 04 09:40:31.538202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532299 2572 flags.go:64] FLAG: --container-log-max-files="5" Aug 04 09:40:31.538202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532303 2572 flags.go:64] FLAG: --container-log-max-size="10Mi" Aug 04 09:40:31.538202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532306 2572 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Aug 04 09:40:31.538202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532309 2572 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Aug 04 09:40:31.538202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532312 2572 flags.go:64] FLAG: --containerd-namespace="k8s.io" Aug 04 09:40:31.538202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532315 2572 flags.go:64] FLAG: --contention-profiling="false" Aug 04 09:40:31.538202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532318 2572 flags.go:64] FLAG: --cpu-cfs-quota="true" Aug 04 09:40:31.538202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532321 2572 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Aug 04 09:40:31.538202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532324 2572 flags.go:64] FLAG: --cpu-manager-policy="none" Aug 04 09:40:31.538202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532327 2572 flags.go:64] FLAG: --cpu-manager-policy-options="" Aug 04 09:40:31.538202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532331 2572 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Aug 04 09:40:31.538202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532334 2572 flags.go:64] FLAG: --enable-controller-attach-detach="true" Aug 04 09:40:31.538202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532337 2572 flags.go:64] FLAG: --enable-debugging-handlers="true" Aug 04 09:40:31.538202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532339 2572 flags.go:64] FLAG: --enable-load-reader="false" Aug 04 09:40:31.538202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532342 2572 flags.go:64] FLAG: --enable-server="true" Aug 04 09:40:31.538202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532345 2572 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Aug 04 09:40:31.538202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532349 2572 flags.go:64] FLAG: --event-burst="100" Aug 04 09:40:31.538202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532352 2572 flags.go:64] FLAG: --event-qps="50" Aug 04 09:40:31.538202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532355 2572 flags.go:64] FLAG: --event-storage-age-limit="default=0" Aug 04 09:40:31.538202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532358 2572 flags.go:64] FLAG: --event-storage-event-limit="default=0" Aug 04 09:40:31.538202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532361 2572 flags.go:64] FLAG: --eviction-hard="" Aug 04 09:40:31.538202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532366 2572 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Aug 04 09:40:31.538202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532369 2572 flags.go:64] FLAG: --eviction-minimum-reclaim="" Aug 04 09:40:31.538202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532372 2572 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Aug 04 09:40:31.538202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532376 2572 flags.go:64] FLAG: --eviction-soft="" Aug 04 09:40:31.538862 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532379 2572 flags.go:64] FLAG: --eviction-soft-grace-period="" Aug 04 09:40:31.538862 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532382 2572 flags.go:64] FLAG: --exit-on-lock-contention="false" Aug 04 09:40:31.538862 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532385 2572 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Aug 04 09:40:31.538862 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532388 2572 flags.go:64] FLAG: --experimental-mounter-path="" Aug 04 09:40:31.538862 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532391 2572 flags.go:64] FLAG: --fail-cgroupv1="false" Aug 04 09:40:31.538862 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532394 2572 flags.go:64] FLAG: --fail-swap-on="true" Aug 04 09:40:31.538862 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532397 2572 flags.go:64] FLAG: --feature-gates="" Aug 04 09:40:31.538862 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532401 2572 flags.go:64] FLAG: --file-check-frequency="20s" Aug 04 09:40:31.538862 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532404 2572 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Aug 04 09:40:31.538862 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532407 2572 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Aug 04 09:40:31.538862 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532410 2572 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Aug 04 09:40:31.538862 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532413 2572 flags.go:64] FLAG: --healthz-port="10248" Aug 04 09:40:31.538862 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532416 2572 flags.go:64] FLAG: --help="false" Aug 04 09:40:31.538862 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532419 2572 flags.go:64] FLAG: --hostname-override="ip-10-0-129-241.ec2.internal" Aug 04 09:40:31.538862 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532422 2572 flags.go:64] FLAG: --housekeeping-interval="10s" Aug 04 09:40:31.538862 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532425 2572 flags.go:64] FLAG: --http-check-frequency="20s" Aug 04 09:40:31.538862 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532428 2572 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Aug 04 09:40:31.538862 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532432 2572 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Aug 04 09:40:31.538862 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532435 2572 flags.go:64] FLAG: --image-gc-high-threshold="85" Aug 04 09:40:31.538862 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532438 2572 flags.go:64] FLAG: --image-gc-low-threshold="80" Aug 04 09:40:31.538862 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532441 2572 flags.go:64] FLAG: --image-service-endpoint="" Aug 04 09:40:31.538862 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532443 2572 flags.go:64] FLAG: --kernel-memcg-notification="false" Aug 04 09:40:31.538862 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532446 2572 flags.go:64] FLAG: --kube-api-burst="100" Aug 04 09:40:31.539478 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532449 2572 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Aug 04 09:40:31.539478 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532452 2572 flags.go:64] FLAG: --kube-api-qps="50" Aug 04 09:40:31.539478 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532455 2572 flags.go:64] FLAG: --kube-reserved="" Aug 04 09:40:31.539478 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532458 2572 flags.go:64] FLAG: --kube-reserved-cgroup="" Aug 04 09:40:31.539478 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532461 2572 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Aug 04 09:40:31.539478 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532465 2572 flags.go:64] FLAG: --kubelet-cgroups="" Aug 04 09:40:31.539478 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532468 2572 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Aug 04 09:40:31.539478 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532471 2572 flags.go:64] FLAG: --lock-file="" Aug 04 09:40:31.539478 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532474 2572 flags.go:64] FLAG: --log-cadvisor-usage="false" Aug 04 09:40:31.539478 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532478 2572 flags.go:64] FLAG: --log-flush-frequency="5s" Aug 04 09:40:31.539478 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532481 2572 flags.go:64] FLAG: --log-json-info-buffer-size="0" Aug 04 09:40:31.539478 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532486 2572 flags.go:64] FLAG: --log-json-split-stream="false" Aug 04 09:40:31.539478 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532489 2572 flags.go:64] FLAG: --log-text-info-buffer-size="0" Aug 04 09:40:31.539478 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532492 2572 flags.go:64] FLAG: --log-text-split-stream="false" Aug 04 09:40:31.539478 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532495 2572 flags.go:64] FLAG: --logging-format="text" Aug 04 09:40:31.539478 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532498 2572 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Aug 04 09:40:31.539478 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532501 2572 flags.go:64] FLAG: --make-iptables-util-chains="true" Aug 04 09:40:31.539478 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532504 2572 flags.go:64] FLAG: --manifest-url="" Aug 04 09:40:31.539478 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532507 2572 flags.go:64] FLAG: --manifest-url-header="" Aug 04 09:40:31.539478 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532511 2572 flags.go:64] FLAG: --max-housekeeping-interval="15s" Aug 04 09:40:31.539478 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532514 2572 flags.go:64] FLAG: --max-open-files="1000000" Aug 04 09:40:31.539478 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532518 2572 flags.go:64] FLAG: --max-pods="110" Aug 04 09:40:31.539478 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532521 2572 flags.go:64] FLAG: --maximum-dead-containers="-1" Aug 04 09:40:31.539478 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532524 2572 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Aug 04 09:40:31.539478 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532527 2572 flags.go:64] FLAG: --memory-manager-policy="None" Aug 04 09:40:31.540114 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532530 2572 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Aug 04 09:40:31.540114 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532533 2572 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Aug 04 09:40:31.540114 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532536 2572 flags.go:64] FLAG: --node-ip="0.0.0.0" Aug 04 09:40:31.540114 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532540 2572 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Aug 04 09:40:31.540114 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532546 2572 flags.go:64] FLAG: --node-status-max-images="50" Aug 04 09:40:31.540114 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532549 2572 flags.go:64] FLAG: --node-status-update-frequency="10s" Aug 04 09:40:31.540114 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532553 2572 flags.go:64] FLAG: --oom-score-adj="-999" Aug 04 09:40:31.540114 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532556 2572 flags.go:64] FLAG: --pod-cidr="" Aug 04 09:40:31.540114 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532558 2572 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8" Aug 04 09:40:31.540114 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532564 2572 flags.go:64] FLAG: --pod-manifest-path="" Aug 04 09:40:31.540114 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532567 2572 flags.go:64] FLAG: --pod-max-pids="-1" Aug 04 09:40:31.540114 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532570 2572 flags.go:64] FLAG: --pods-per-core="0" Aug 04 09:40:31.540114 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532574 2572 flags.go:64] FLAG: --port="10250" Aug 04 09:40:31.540114 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532578 2572 flags.go:64] FLAG: --protect-kernel-defaults="false" Aug 04 09:40:31.540114 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532581 2572 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0075293f66bf475d6" Aug 04 09:40:31.540114 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532584 2572 flags.go:64] FLAG: --qos-reserved="" Aug 04 09:40:31.540114 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532587 2572 flags.go:64] FLAG: --read-only-port="10255" Aug 04 09:40:31.540114 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532597 2572 flags.go:64] FLAG: --register-node="true" Aug 04 09:40:31.540114 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532600 2572 flags.go:64] FLAG: --register-with-taints="" Aug 04 09:40:31.540114 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532604 2572 flags.go:64] FLAG: --registry-burst="10" Aug 04 09:40:31.540114 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532620 2572 flags.go:64] FLAG: --registry-qps="5" Aug 04 09:40:31.540114 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532631 2572 flags.go:64] FLAG: --reserved-cpus="" Aug 04 09:40:31.540114 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532635 2572 flags.go:64] FLAG: --reserved-memory="" Aug 04 09:40:31.540114 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532639 2572 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Aug 04 09:40:31.540114 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532642 2572 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Aug 04 09:40:31.540719 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532645 2572 flags.go:64] FLAG: --rotate-certificates="false" Aug 04 09:40:31.540719 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532648 2572 flags.go:64] FLAG: --rotate-server-certificates="false" Aug 04 09:40:31.540719 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532651 2572 flags.go:64] FLAG: --runonce="false" Aug 04 09:40:31.540719 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532654 2572 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Aug 04 09:40:31.540719 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532658 2572 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Aug 04 09:40:31.540719 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532660 2572 flags.go:64] FLAG: --seccomp-default="false" Aug 04 09:40:31.540719 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532663 2572 flags.go:64] FLAG: --serialize-image-pulls="true" Aug 04 09:40:31.540719 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532666 2572 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Aug 04 09:40:31.540719 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532669 2572 flags.go:64] FLAG: --storage-driver-db="cadvisor" Aug 04 09:40:31.540719 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532672 2572 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Aug 04 09:40:31.540719 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532676 2572 flags.go:64] FLAG: --storage-driver-password="root" Aug 04 09:40:31.540719 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532679 2572 flags.go:64] FLAG: --storage-driver-secure="false" Aug 04 09:40:31.540719 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532681 2572 flags.go:64] FLAG: --storage-driver-table="stats" Aug 04 09:40:31.540719 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532684 2572 flags.go:64] FLAG: --storage-driver-user="root" Aug 04 09:40:31.540719 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532687 2572 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Aug 04 09:40:31.540719 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532690 2572 flags.go:64] FLAG: --sync-frequency="1m0s" Aug 04 09:40:31.540719 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532693 2572 flags.go:64] FLAG: --system-cgroups="" Aug 04 09:40:31.540719 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532696 2572 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Aug 04 09:40:31.540719 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532701 2572 flags.go:64] FLAG: --system-reserved-cgroup="" Aug 04 09:40:31.540719 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532705 2572 flags.go:64] FLAG: --tls-cert-file="" Aug 04 09:40:31.540719 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532708 2572 flags.go:64] FLAG: --tls-cipher-suites="[]" Aug 04 09:40:31.540719 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532716 2572 flags.go:64] FLAG: --tls-min-version="" Aug 04 09:40:31.540719 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532718 2572 flags.go:64] FLAG: --tls-private-key-file="" Aug 04 09:40:31.540719 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532721 2572 flags.go:64] FLAG: --topology-manager-policy="none" Aug 04 09:40:31.540719 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532724 2572 flags.go:64] FLAG: --topology-manager-policy-options="" Aug 04 09:40:31.541327 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532728 2572 flags.go:64] FLAG: --topology-manager-scope="container" Aug 04 09:40:31.541327 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532731 2572 flags.go:64] FLAG: --v="2" Aug 04 09:40:31.541327 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532735 2572 flags.go:64] FLAG: --version="false" Aug 04 09:40:31.541327 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532739 2572 flags.go:64] FLAG: --vmodule="" Aug 04 09:40:31.541327 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532743 2572 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Aug 04 09:40:31.541327 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.532746 2572 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Aug 04 09:40:31.541327 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532875 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 09:40:31.541327 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532879 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 09:40:31.541327 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532882 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 09:40:31.541327 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532885 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 09:40:31.541327 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532888 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 09:40:31.541327 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532891 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 09:40:31.541327 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532893 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 09:40:31.541327 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532896 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 09:40:31.541327 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532898 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 09:40:31.541327 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532901 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 09:40:31.541327 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532903 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 09:40:31.541327 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532907 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 09:40:31.541327 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532911 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 09:40:31.541327 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532913 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 09:40:31.541835 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532916 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 09:40:31.541835 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532919 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 09:40:31.541835 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532922 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 09:40:31.541835 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532925 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 09:40:31.541835 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532928 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 09:40:31.541835 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532931 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 09:40:31.541835 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532936 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 09:40:31.541835 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532939 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 09:40:31.541835 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532943 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 09:40:31.541835 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532946 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 09:40:31.541835 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532949 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 09:40:31.541835 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532951 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 09:40:31.541835 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532953 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 09:40:31.541835 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532957 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 09:40:31.541835 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532960 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 09:40:31.541835 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532962 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 09:40:31.541835 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532965 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 09:40:31.541835 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532967 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 09:40:31.541835 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532970 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 09:40:31.541835 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532972 2572 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 09:40:31.542358 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532974 2572 feature_gate.go:328] unrecognized feature gate: Example Aug 04 09:40:31.542358 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532977 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 09:40:31.542358 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532979 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 09:40:31.542358 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532982 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 09:40:31.542358 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532984 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 09:40:31.542358 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532987 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 09:40:31.542358 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532990 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 09:40:31.542358 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532992 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 09:40:31.542358 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532995 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 09:40:31.542358 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.532997 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 09:40:31.542358 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533000 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 09:40:31.542358 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533002 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 09:40:31.542358 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533005 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 09:40:31.542358 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533007 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 09:40:31.542358 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533010 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 09:40:31.542358 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533012 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 09:40:31.542358 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533015 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 09:40:31.542358 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533017 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 09:40:31.542358 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533022 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 09:40:31.542358 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533024 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 09:40:31.542864 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533029 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 09:40:31.542864 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533031 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 09:40:31.542864 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533034 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 09:40:31.542864 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533037 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 09:40:31.542864 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533039 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 09:40:31.542864 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533042 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 09:40:31.542864 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533051 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 09:40:31.542864 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533054 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 09:40:31.542864 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533057 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 09:40:31.542864 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533059 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 09:40:31.542864 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533062 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 09:40:31.542864 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533064 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 09:40:31.542864 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533067 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 09:40:31.542864 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533069 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 09:40:31.542864 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533072 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 09:40:31.542864 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533074 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 09:40:31.542864 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533076 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 09:40:31.542864 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533079 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 09:40:31.542864 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533081 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 09:40:31.542864 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533084 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 09:40:31.543340 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533087 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 09:40:31.543340 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533090 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 09:40:31.543340 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533094 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 09:40:31.543340 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533097 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 09:40:31.543340 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533099 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 09:40:31.543340 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533102 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 09:40:31.543340 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533104 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 09:40:31.543340 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533107 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 09:40:31.543340 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533109 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 09:40:31.543340 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533112 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 09:40:31.543340 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533116 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 09:40:31.543340 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533119 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 09:40:31.543340 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533123 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 09:40:31.543340 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533125 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 09:40:31.543340 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533129 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 09:40:31.543340 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533133 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 09:40:31.543340 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533136 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 09:40:31.543340 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533138 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 09:40:31.543340 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533141 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 09:40:31.543878 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533149 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 09:40:31.543878 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533152 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 09:40:31.543878 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533154 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 09:40:31.543878 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533157 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 09:40:31.543878 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533159 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 09:40:31.543878 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.533162 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 09:40:31.543878 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.533748 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 09:40:31.543878 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.540365 2572 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Aug 04 09:40:31.543878 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.540383 2572 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Aug 04 09:40:31.543878 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.540404 2572 watchdog_linux.go:95] "Systemd watchdog is not enabled" Aug 04 09:40:31.543878 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.540410 2572 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Aug 04 09:40:31.543878 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540467 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 09:40:31.543878 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540472 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 09:40:31.543878 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540476 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 09:40:31.543878 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540479 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 09:40:31.544269 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540482 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 09:40:31.544269 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540486 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 09:40:31.544269 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540490 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 09:40:31.544269 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540493 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 09:40:31.544269 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540497 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 09:40:31.544269 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540499 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 09:40:31.544269 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540502 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 09:40:31.544269 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540505 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 09:40:31.544269 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540508 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 09:40:31.544269 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540511 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 09:40:31.544269 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540514 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 09:40:31.544269 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540517 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 09:40:31.544269 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540519 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 09:40:31.544269 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540522 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 09:40:31.544269 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540525 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 09:40:31.544269 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540528 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 09:40:31.544269 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540530 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 09:40:31.544269 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540533 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 09:40:31.544269 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540536 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 09:40:31.544747 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540538 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 09:40:31.544747 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540541 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 09:40:31.544747 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540543 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 09:40:31.544747 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540546 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 09:40:31.544747 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540548 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 09:40:31.544747 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540551 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 09:40:31.544747 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540555 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 09:40:31.544747 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540557 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 09:40:31.544747 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540560 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 09:40:31.544747 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540563 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 09:40:31.544747 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540565 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 09:40:31.544747 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540568 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 09:40:31.544747 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540570 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 09:40:31.544747 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540573 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 09:40:31.544747 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540575 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 09:40:31.544747 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540578 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 09:40:31.544747 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540581 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 09:40:31.544747 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540583 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 09:40:31.544747 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540586 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 09:40:31.544747 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540588 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 09:40:31.545239 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540591 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 09:40:31.545239 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540593 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 09:40:31.545239 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540596 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 09:40:31.545239 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540599 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 09:40:31.545239 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540601 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 09:40:31.545239 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540604 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 09:40:31.545239 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540607 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 09:40:31.545239 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540609 2572 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 09:40:31.545239 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540612 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 09:40:31.545239 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540614 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 09:40:31.545239 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540617 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 09:40:31.545239 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540620 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 09:40:31.545239 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540622 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 09:40:31.545239 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540624 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 09:40:31.545239 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540627 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 09:40:31.545239 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540630 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 09:40:31.545239 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540632 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 09:40:31.545239 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540637 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 09:40:31.545239 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540640 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 09:40:31.545239 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540643 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 09:40:31.545733 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540646 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 09:40:31.545733 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540649 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 09:40:31.545733 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540652 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 09:40:31.545733 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540654 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 09:40:31.545733 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540657 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 09:40:31.545733 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540659 2572 feature_gate.go:328] unrecognized feature gate: Example Aug 04 09:40:31.545733 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540662 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 09:40:31.545733 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540664 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 09:40:31.545733 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540667 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 09:40:31.545733 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540669 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 09:40:31.545733 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540672 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 09:40:31.545733 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540675 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 09:40:31.545733 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540679 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 09:40:31.545733 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540682 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 09:40:31.545733 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540685 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 09:40:31.545733 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540688 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 09:40:31.545733 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540691 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 09:40:31.545733 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540693 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 09:40:31.545733 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540696 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 09:40:31.545733 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540698 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 09:40:31.546280 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540701 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 09:40:31.546280 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540703 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 09:40:31.546280 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540706 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 09:40:31.546280 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540709 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 09:40:31.546280 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540711 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 09:40:31.546280 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540713 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 09:40:31.546280 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540716 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 09:40:31.546280 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540719 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 09:40:31.546280 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540721 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 09:40:31.546280 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540724 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 09:40:31.546280 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540728 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 09:40:31.546280 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540730 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 09:40:31.546280 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540733 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 09:40:31.546280 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540736 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 09:40:31.546280 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540739 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 09:40:31.546280 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540741 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 09:40:31.546680 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.540746 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 09:40:31.546680 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540876 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 09:40:31.546680 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540883 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 09:40:31.546680 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540886 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 09:40:31.546680 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540890 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 09:40:31.546680 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540893 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 09:40:31.546680 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540896 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 09:40:31.546680 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540898 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 09:40:31.546680 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540901 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 09:40:31.546680 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540903 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 09:40:31.546680 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540906 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 09:40:31.546680 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540908 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 09:40:31.546680 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540911 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 09:40:31.546680 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540914 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 09:40:31.547061 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540916 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 09:40:31.547061 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540919 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 09:40:31.547061 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540921 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 09:40:31.547061 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540924 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 09:40:31.547061 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540926 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 09:40:31.547061 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540929 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 09:40:31.547061 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540931 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 09:40:31.547061 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540934 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 09:40:31.547061 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540936 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 09:40:31.547061 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540939 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 09:40:31.547061 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540941 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 09:40:31.547061 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540944 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 09:40:31.547061 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540946 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 09:40:31.547061 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540949 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 09:40:31.547061 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540951 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 09:40:31.547061 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540955 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 09:40:31.547061 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540957 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 09:40:31.547061 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540960 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 09:40:31.547061 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540962 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 09:40:31.547061 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540964 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 09:40:31.547596 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540967 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 09:40:31.547596 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540969 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 09:40:31.547596 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540972 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 09:40:31.547596 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540974 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 09:40:31.547596 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540977 2572 feature_gate.go:328] unrecognized feature gate: Example Aug 04 09:40:31.547596 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540980 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 09:40:31.547596 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540984 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 09:40:31.547596 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540987 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 09:40:31.547596 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540989 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 09:40:31.547596 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540992 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 09:40:31.547596 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540995 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 09:40:31.547596 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.540998 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 09:40:31.547596 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541001 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 09:40:31.547596 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541004 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 09:40:31.547596 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541006 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 09:40:31.547596 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541009 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 09:40:31.547596 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541011 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 09:40:31.547596 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541014 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 09:40:31.547596 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541016 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 09:40:31.547596 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541019 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 09:40:31.548282 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541021 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 09:40:31.548282 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541024 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 09:40:31.548282 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541026 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 09:40:31.548282 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541029 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 09:40:31.548282 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541032 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 09:40:31.548282 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541036 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 09:40:31.548282 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541040 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 09:40:31.548282 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541043 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 09:40:31.548282 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541046 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 09:40:31.548282 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541048 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 09:40:31.548282 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541051 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 09:40:31.548282 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541054 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 09:40:31.548282 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541056 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 09:40:31.548282 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541059 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 09:40:31.548282 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541061 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 09:40:31.548282 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541064 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 09:40:31.548282 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541066 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 09:40:31.548282 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541069 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 09:40:31.548282 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541071 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 09:40:31.549085 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541074 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 09:40:31.549085 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541076 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 09:40:31.549085 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541079 2572 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 09:40:31.549085 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541082 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 09:40:31.549085 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541084 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 09:40:31.549085 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541087 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 09:40:31.549085 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541090 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 09:40:31.549085 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541092 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 09:40:31.549085 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541094 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 09:40:31.549085 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541097 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 09:40:31.549085 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541099 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 09:40:31.549085 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541102 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 09:40:31.549085 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541104 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 09:40:31.549085 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541107 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 09:40:31.549085 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541109 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 09:40:31.549085 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541112 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 09:40:31.549085 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541115 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 09:40:31.549085 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541117 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 09:40:31.549085 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541120 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 09:40:31.549085 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541123 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 09:40:31.549708 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541125 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 09:40:31.549708 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541127 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 09:40:31.549708 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541131 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 09:40:31.549708 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541133 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 09:40:31.549708 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541135 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 09:40:31.549708 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541138 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 09:40:31.549708 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:31.541140 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 09:40:31.549708 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.541145 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 09:40:31.549708 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.541261 2572 server.go:962] "Client rotation is on, will bootstrap in background" Aug 04 09:40:31.549708 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.543221 2572 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Aug 04 09:40:31.549708 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.544604 2572 server.go:1019] "Starting client certificate rotation" Aug 04 09:40:31.549708 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.544707 2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Aug 04 09:40:31.549708 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.545552 2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Aug 04 09:40:31.550148 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.547085 2572 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Aug 04 09:40:31.550402 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.550383 2572 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Aug 04 09:40:31.559299 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.559276 2572 log.go:25] "Validated CRI v1 runtime API" Aug 04 09:40:31.561377 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.561359 2572 log.go:25] "Validated CRI v1 image API" Aug 04 09:40:31.562964 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.562948 2572 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Aug 04 09:40:31.565323 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.565298 2572 fs.go:135] Filesystem UUIDs: map[6f0364f1-99c6-45d9-a595-98c64194516d:/dev/nvme0n1p3 75a487be-a120-496a-a631-da9593417223:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Aug 04 09:40:31.565398 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.565321 2572 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Aug 04 09:40:31.571079 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.570953 2572 manager.go:216] Machine: {Timestamp:2026-08-04 09:40:31.569150756 +0000 UTC m=+0.313285165 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100108 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec221a9638cce0aa5960a0ecae3bce77 SystemUUID:ec221a96-38cc-e0aa-5960-a0ecae3bce77 BootID:01a5f157-5184-479b-a18b-f58e3b6dd51f Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:77:6c:44:d7:a7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:77:6c:44:d7:a7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:46:81:15:61:a8:86 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Aug 04 09:40:31.571079 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.571068 2572 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Aug 04 09:40:31.571262 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.571191 2572 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Aug 04 09:40:31.573912 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.573880 2572 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Aug 04 09:40:31.574090 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.573921 2572 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-241.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Aug 04 09:40:31.574730 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.574718 2572 topology_manager.go:138] "Creating topology manager with none policy" Aug 04 09:40:31.574781 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.574736 2572 container_manager_linux.go:309] "Creating device plugin manager" Aug 04 09:40:31.574781 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.574748 2572 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Aug 04 09:40:31.575446 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.575434 2572 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Aug 04 09:40:31.576152 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.576140 2572 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Aug 04 09:40:31.579605 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.579590 2572 state_mem.go:36] "Initialized new in-memory state store" Aug 04 09:40:31.579748 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.579737 2572 server.go:1240] "Using root directory" path="/var/lib/kubelet" Aug 04 09:40:31.580398 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.580388 2572 kubelet.go:486] "Attempting to sync node with API server" Aug 04 09:40:31.580459 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.580406 2572 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Aug 04 09:40:31.580459 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.580425 2572 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Aug 04 09:40:31.580459 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.580439 2572 kubelet.go:398] "Adding apiserver pod source" Aug 04 09:40:31.580459 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.580452 2572 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Aug 04 09:40:31.580955 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.580940 2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Aug 04 09:40:31.581036 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.580963 2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Aug 04 09:40:31.583821 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.583778 2572 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Aug 04 09:40:31.584690 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.584673 2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Aug 04 09:40:31.585671 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.585639 2572 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Aug 04 09:40:31.585751 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.585682 2572 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Aug 04 09:40:31.586903 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.586885 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Aug 04 09:40:31.586980 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.586913 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Aug 04 09:40:31.586980 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.586930 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Aug 04 09:40:31.586980 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.586943 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Aug 04 09:40:31.586980 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.586957 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Aug 04 09:40:31.586980 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.586966 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Aug 04 09:40:31.586980 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.586980 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Aug 04 09:40:31.587194 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.586995 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Aug 04 09:40:31.587194 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.587020 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Aug 04 09:40:31.587194 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.587031 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Aug 04 09:40:31.587194 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.587046 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Aug 04 09:40:31.587194 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.587057 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Aug 04 09:40:31.588393 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.588371 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Aug 04 09:40:31.588467 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.588454 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Aug 04 09:40:31.591775 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.591758 2572 server.go:1268] "Started kubelet" Aug 04 09:40:31.591875 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.591843 2572 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Aug 04 09:40:31.591999 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.591934 2572 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Aug 04 09:40:31.592063 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.592021 2572 server_v1.go:49] "podresources" method="list" useActivePods=true Aug 04 09:40:31.592994 ip-10-0-129-241 systemd[1]: Started Kubernetes Kubelet. Aug 04 09:40:31.593538 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.593428 2572 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Aug 04 09:40:31.594123 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.594110 2572 server.go:310] "Adding debug handlers to kubelet server" Aug 04 09:40:31.598470 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:31.598451 2572 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Aug 04 09:40:31.599300 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.599284 2572 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-241.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Aug 04 09:40:31.599347 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:31.599285 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Aug 04 09:40:31.600022 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:31.600002 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-241.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Aug 04 09:40:31.600397 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.600381 2572 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Aug 04 09:40:31.600468 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.600430 2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Aug 04 09:40:31.600542 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:31.600507 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-241.ec2.internal\" not found" Aug 04 09:40:31.601096 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.601072 2572 volume_manager.go:311] "The desired_state_of_world populator starts" Aug 04 09:40:31.601096 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.601096 2572 volume_manager.go:313] "Starting Kubelet Volume Manager" Aug 04 09:40:31.601239 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.601075 2572 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Aug 04 09:40:31.601239 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.601211 2572 reconstruct.go:97] "Volume reconstruction finished" Aug 04 09:40:31.601239 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.601218 2572 reconciler.go:29] "Reconciler: start to sync state" Aug 04 09:40:31.601472 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.601438 2572 factory.go:55] Registering systemd factory Aug 04 09:40:31.601472 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.601457 2572 factory.go:223] Registration of the systemd container factory successfully Aug 04 09:40:31.601710 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.601683 2572 factory.go:153] Registering CRI-O factory Aug 04 09:40:31.601710 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.601702 2572 factory.go:223] Registration of the crio container factory successfully Aug 04 09:40:31.601809 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.601768 2572 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Aug 04 09:40:31.601850 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.601826 2572 factory.go:103] Registering Raw factory Aug 04 09:40:31.601850 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.601840 2572 manager.go:1196] Started watching for new ooms in manager Aug 04 09:40:31.602556 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.602426 2572 manager.go:349] Starting recovery of all containers Aug 04 09:40:31.604427 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:31.604394 2572 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-241.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Aug 04 09:40:31.604616 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:31.604597 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Aug 04 09:40:31.607320 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:31.606251 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-241.ec2.internal.18c890c5d3f9f030 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-241.ec2.internal,UID:ip-10-0-129-241.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-241.ec2.internal,},FirstTimestamp:2026-08-04 09:40:31.591731248 +0000 UTC m=+0.335865658,LastTimestamp:2026-08-04 09:40:31.591731248 +0000 UTC m=+0.335865658,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-241.ec2.internal,}" Aug 04 09:40:31.613259 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.613237 2572 manager.go:354] Recovery completed Aug 04 09:40:31.618700 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.618682 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 09:40:31.620456 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.620436 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-241.ec2.internal" event="NodeHasSufficientMemory" Aug 04 09:40:31.620524 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.620470 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-241.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 09:40:31.620524 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.620483 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-241.ec2.internal" event="NodeHasSufficientPID" Aug 04 09:40:31.621021 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.621009 2572 cpu_manager.go:222] "Starting CPU manager" policy="none" Aug 04 09:40:31.621021 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.621018 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 09:40:31.621108 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.621020 2572 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Aug 04 09:40:31.621108 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.621081 2572 state_mem.go:36] "Initialized new in-memory state store" Aug 04 09:40:31.621668 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.621654 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-241.ec2.internal" event="NodeHasSufficientMemory" Aug 04 09:40:31.621714 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.621681 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-241.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 09:40:31.621714 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.621694 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-241.ec2.internal" event="NodeHasSufficientPID" Aug 04 09:40:31.622644 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.622633 2572 policy_none.go:49] "None policy: Start" Aug 04 09:40:31.622691 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.622649 2572 memory_manager.go:187] "Starting memorymanager" policy="None" Aug 04 09:40:31.622691 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.622660 2572 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Aug 04 09:40:31.623408 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:31.623310 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-241.ec2.internal.18c890c5d5b03d49 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-241.ec2.internal,UID:ip-10-0-129-241.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-241.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-241.ec2.internal,},FirstTimestamp:2026-08-04 09:40:31.620455753 +0000 UTC m=+0.364590166,LastTimestamp:2026-08-04 09:40:31.620455753 +0000 UTC m=+0.364590166,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-241.ec2.internal,}" Aug 04 09:40:31.623646 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.623636 2572 policy_none.go:47] "Start" Aug 04 09:40:31.638675 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:31.638590 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-241.ec2.internal.18c890c5d5b09154 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-241.ec2.internal,UID:ip-10-0-129-241.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-241.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-241.ec2.internal,},FirstTimestamp:2026-08-04 09:40:31.620477268 +0000 UTC m=+0.364611681,LastTimestamp:2026-08-04 09:40:31.620477268 +0000 UTC m=+0.364611681,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-241.ec2.internal,}" Aug 04 09:40:31.646071 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:31.645995 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-241.ec2.internal.18c890c5d5b0b9ce default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-241.ec2.internal,UID:ip-10-0-129-241.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-241.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-241.ec2.internal,},FirstTimestamp:2026-08-04 09:40:31.62048763 +0000 UTC m=+0.364622040,LastTimestamp:2026-08-04 09:40:31.62048763 +0000 UTC m=+0.364622040,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-241.ec2.internal,}" Aug 04 09:40:31.657540 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:31.657461 2572 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-241.ec2.internal.18c890c5d5b03d49\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-241.ec2.internal.18c890c5d5b03d49 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-241.ec2.internal,UID:ip-10-0-129-241.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-241.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-241.ec2.internal,},FirstTimestamp:2026-08-04 09:40:31.620455753 +0000 UTC m=+0.364590166,LastTimestamp:2026-08-04 09:40:31.621668875 +0000 UTC m=+0.365803288,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-241.ec2.internal,}" Aug 04 09:40:31.665585 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.665567 2572 manager.go:337] "Starting Device Plugin manager" Aug 04 09:40:31.665686 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:31.665601 2572 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Aug 04 09:40:31.665686 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.665612 2572 server.go:87] "Starting device plugin registration server" Aug 04 09:40:31.665904 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.665890 2572 eviction_manager.go:189] "Eviction manager: starting control loop" Aug 04 09:40:31.665987 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.665906 2572 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Aug 04 09:40:31.666028 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.665996 2572 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Aug 04 09:40:31.666315 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.666155 2572 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Aug 04 09:40:31.666315 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.666169 2572 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Aug 04 09:40:31.667031 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:31.666964 2572 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Aug 04 09:40:31.667185 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:31.667156 2572 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-241.ec2.internal\" not found" Aug 04 09:40:31.668129 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:31.668056 2572 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-241.ec2.internal.18c890c5d5b09154\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-241.ec2.internal.18c890c5d5b09154 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-241.ec2.internal,UID:ip-10-0-129-241.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-241.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-241.ec2.internal,},FirstTimestamp:2026-08-04 09:40:31.620477268 +0000 UTC m=+0.364611681,LastTimestamp:2026-08-04 09:40:31.621687921 +0000 UTC m=+0.365822335,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-241.ec2.internal,}" Aug 04 09:40:31.677664 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.677646 2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-77gr5" Aug 04 09:40:31.683034 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:31.682957 2572 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-241.ec2.internal.18c890c5d5b0b9ce\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-241.ec2.internal.18c890c5d5b0b9ce default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-241.ec2.internal,UID:ip-10-0-129-241.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-241.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-241.ec2.internal,},FirstTimestamp:2026-08-04 09:40:31.62048763 +0000 UTC m=+0.364622040,LastTimestamp:2026-08-04 09:40:31.621698765 +0000 UTC m=+0.365833174,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-241.ec2.internal,}" Aug 04 09:40:31.696398 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:31.696328 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-241.ec2.internal.18c890c5d9132aff default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-241.ec2.internal,UID:ip-10-0-129-241.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-129-241.ec2.internal,},FirstTimestamp:2026-08-04 09:40:31.677270783 +0000 UTC m=+0.421405181,LastTimestamp:2026-08-04 09:40:31.677270783 +0000 UTC m=+0.421405181,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-241.ec2.internal,}" Aug 04 09:40:31.716855 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.701005 2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-77gr5" Aug 04 09:40:31.767061 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.767026 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 09:40:31.768022 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.768004 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-241.ec2.internal" event="NodeHasSufficientMemory" Aug 04 09:40:31.768104 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.768034 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-241.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 09:40:31.768104 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.768044 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-241.ec2.internal" event="NodeHasSufficientPID" Aug 04 09:40:31.768104 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.768071 2572 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-241.ec2.internal" Aug 04 09:40:31.770529 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.770504 2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Aug 04 09:40:31.771763 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.771742 2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Aug 04 09:40:31.771853 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.771790 2572 status_manager.go:244] "Starting to sync pod status with apiserver" Aug 04 09:40:31.771853 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.771836 2572 kubelet.go:2448] "Starting kubelet main sync loop" Aug 04 09:40:31.771941 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:31.771877 2572 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Aug 04 09:40:31.781291 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.781269 2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 09:40:31.788563 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.788543 2572 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-241.ec2.internal" Aug 04 09:40:31.788655 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:31.788571 2572 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-241.ec2.internal\": node \"ip-10-0-129-241.ec2.internal\" not found" Aug 04 09:40:31.870338 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:31.870303 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-241.ec2.internal\" not found" Aug 04 09:40:31.872907 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.872883 2572 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-241.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-241.ec2.internal"] Aug 04 09:40:31.873017 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.872977 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 09:40:31.873897 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.873878 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-241.ec2.internal" event="NodeHasSufficientMemory" Aug 04 09:40:31.874015 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.873913 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-241.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 09:40:31.874015 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.873937 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-241.ec2.internal" event="NodeHasSufficientPID" Aug 04 09:40:31.875933 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.875917 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 09:40:31.876042 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.876029 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-241.ec2.internal" Aug 04 09:40:31.876089 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.876056 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 09:40:31.876674 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.876660 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-241.ec2.internal" event="NodeHasSufficientMemory" Aug 04 09:40:31.876740 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.876684 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-241.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 09:40:31.876740 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.876695 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-241.ec2.internal" event="NodeHasSufficientPID" Aug 04 09:40:31.876825 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.876662 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-241.ec2.internal" event="NodeHasSufficientMemory" Aug 04 09:40:31.876825 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.876761 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-241.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 09:40:31.876825 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.876770 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-241.ec2.internal" event="NodeHasSufficientPID" Aug 04 09:40:31.878764 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.878747 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-241.ec2.internal" Aug 04 09:40:31.878868 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.878773 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 09:40:31.879586 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.879564 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-241.ec2.internal" event="NodeHasSufficientMemory" Aug 04 09:40:31.879656 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.879591 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-241.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 09:40:31.879656 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.879606 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-241.ec2.internal" event="NodeHasSufficientPID" Aug 04 09:40:31.895452 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:31.895423 2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-241.ec2.internal\" not found" node="ip-10-0-129-241.ec2.internal" Aug 04 09:40:31.898914 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:31.898894 2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-241.ec2.internal\" not found" node="ip-10-0-129-241.ec2.internal" Aug 04 09:40:31.903106 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.903088 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f0c99023fafad27b36808bb52bb184dd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-241.ec2.internal\" (UID: \"f0c99023fafad27b36808bb52bb184dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-241.ec2.internal" Aug 04 09:40:31.903161 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.903113 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0c99023fafad27b36808bb52bb184dd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-241.ec2.internal\" (UID: \"f0c99023fafad27b36808bb52bb184dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-241.ec2.internal" Aug 04 09:40:31.903161 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:31.903130 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/88730f4d5c4cac33f5b85009ff17427c-config\") pod \"kube-apiserver-proxy-ip-10-0-129-241.ec2.internal\" (UID: \"88730f4d5c4cac33f5b85009ff17427c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-241.ec2.internal" Aug 04 09:40:31.970673 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:31.970602 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-241.ec2.internal\" not found" Aug 04 09:40:32.004080 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.004052 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f0c99023fafad27b36808bb52bb184dd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-241.ec2.internal\" (UID: \"f0c99023fafad27b36808bb52bb184dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-241.ec2.internal" Aug 04 09:40:32.004183 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.004081 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0c99023fafad27b36808bb52bb184dd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-241.ec2.internal\" (UID: \"f0c99023fafad27b36808bb52bb184dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-241.ec2.internal" Aug 04 09:40:32.004183 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.004097 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/88730f4d5c4cac33f5b85009ff17427c-config\") pod \"kube-apiserver-proxy-ip-10-0-129-241.ec2.internal\" (UID: \"88730f4d5c4cac33f5b85009ff17427c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-241.ec2.internal" Aug 04 09:40:32.004183 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.004136 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/88730f4d5c4cac33f5b85009ff17427c-config\") pod \"kube-apiserver-proxy-ip-10-0-129-241.ec2.internal\" (UID: \"88730f4d5c4cac33f5b85009ff17427c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-241.ec2.internal" Aug 04 09:40:32.004286 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.004184 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f0c99023fafad27b36808bb52bb184dd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-241.ec2.internal\" (UID: \"f0c99023fafad27b36808bb52bb184dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-241.ec2.internal" Aug 04 09:40:32.004286 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.004202 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f0c99023fafad27b36808bb52bb184dd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-241.ec2.internal\" (UID: \"f0c99023fafad27b36808bb52bb184dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-241.ec2.internal" Aug 04 09:40:32.071453 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:32.071417 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-241.ec2.internal\" not found" Aug 04 09:40:32.172376 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:32.172340 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-241.ec2.internal\" not found" Aug 04 09:40:32.197547 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.197517 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-241.ec2.internal" Aug 04 09:40:32.201279 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.201258 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-241.ec2.internal" Aug 04 09:40:32.272844 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:32.272741 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-241.ec2.internal\" not found" Aug 04 09:40:32.373327 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:32.373299 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-241.ec2.internal\" not found" Aug 04 09:40:32.473844 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:32.473786 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-241.ec2.internal\" not found" Aug 04 09:40:32.490118 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.490094 2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 09:40:32.501547 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.501518 2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-241.ec2.internal" Aug 04 09:40:32.513078 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.513052 2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Aug 04 09:40:32.514939 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.514925 2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-241.ec2.internal" Aug 04 09:40:32.526671 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.526618 2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Aug 04 09:40:32.543807 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.543759 2572 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Aug 04 09:40:32.543953 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.543934 2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Aug 04 09:40:32.543995 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.543975 2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Aug 04 09:40:32.560211 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.560173 2572 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 09:40:32.580725 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.580696 2572 apiserver.go:52] "Watching apiserver" Aug 04 09:40:32.589217 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.589188 2572 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Aug 04 09:40:32.590957 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.590935 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-57rd7","openshift-multus/multus-94bwj","openshift-multus/network-metrics-daemon-pjdn8","kube-system/konnectivity-agent-hgcgb","kube-system/kube-apiserver-proxy-ip-10-0-129-241.ec2.internal","openshift-cluster-node-tuning-operator/tuned-6n7bh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-241.ec2.internal","openshift-multus/multus-additional-cni-plugins-2jnfw","openshift-network-diagnostics/network-check-target-qcql5","openshift-network-operator/iptables-alerter-mbstc","openshift-ovn-kubernetes/ovnkube-node-8sdsr","kube-system/global-pull-secret-syncer-7wl64","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdnhg","openshift-dns/node-resolver-qvwd8"] Aug 04 09:40:32.593083 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.593069 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-57rd7" Aug 04 09:40:32.594993 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.594973 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qvwd8" Aug 04 09:40:32.596848 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.596833 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:40:32.596909 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:32.596889 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wl64" podUID="8f732a38-dafb-42d2-acc6-ff8059c65350" Aug 04 09:40:32.598447 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.598430 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Aug 04 09:40:32.598575 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.598560 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zr5l8\"" Aug 04 09:40:32.598735 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.598719 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hgcgb" Aug 04 09:40:32.600228 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.600212 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Aug 04 09:40:32.600350 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.600334 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Aug 04 09:40:32.600414 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.600405 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Aug 04 09:40:32.600518 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.600501 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Aug 04 09:40:32.600564 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.600509 2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Aug 04 09:40:32.600868 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.600852 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.600944 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.600915 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-hwpqk\"" Aug 04 09:40:32.601648 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.601635 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Aug 04 09:40:32.601717 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.601649 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-86j4v\"" Aug 04 09:40:32.601717 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.601700 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Aug 04 09:40:32.602874 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.602859 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.603599 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.603579 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Aug 04 09:40:32.603673 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.603622 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Aug 04 09:40:32.604070 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.604053 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Aug 04 09:40:32.605377 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.605036 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Aug 04 09:40:32.605377 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.605051 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2jnfw" Aug 04 09:40:32.605377 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.605136 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Aug 04 09:40:32.605377 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.605158 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Aug 04 09:40:32.605377 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.605236 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Aug 04 09:40:32.605377 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.605251 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vm472\"" Aug 04 09:40:32.605787 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.605772 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Aug 04 09:40:32.606083 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.606069 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-z6xsp\"" Aug 04 09:40:32.606863 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.606847 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-etc-sysconfig\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.606934 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.606870 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-sys\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.606934 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.606886 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvdcg\" (UniqueName: \"kubernetes.io/projected/a9ba7a09-0ff2-4657-8bbd-47b0af96a45a-kube-api-access-qvdcg\") pod \"node-ca-57rd7\" (UID: \"a9ba7a09-0ff2-4657-8bbd-47b0af96a45a\") " pod="openshift-image-registry/node-ca-57rd7" Aug 04 09:40:32.606934 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.606900 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-log-socket\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.606934 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.606916 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.607133 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.606947 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7858cfca-73db-4ee1-9882-c13444f112a5-ovn-node-metrics-cert\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.607133 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.606971 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6brcj\" (UniqueName: \"kubernetes.io/projected/7858cfca-73db-4ee1-9882-c13444f112a5-kube-api-access-6brcj\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.607133 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607014 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-etc-kubernetes\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.607133 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607049 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/781f670f-6934-4504-810d-95429f368c7f-konnectivity-ca\") pod \"konnectivity-agent-hgcgb\" (UID: \"781f670f-6934-4504-810d-95429f368c7f\") " pod="kube-system/konnectivity-agent-hgcgb" Aug 04 09:40:32.607133 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607098 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-var-lib-openvswitch\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.607374 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607123 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-host-cni-netd\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.607374 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607161 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-etc-sysctl-conf\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.607374 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607198 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-var-lib-kubelet\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.607374 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607227 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-host\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.607374 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607253 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8f732a38-dafb-42d2-acc6-ff8059c65350-kubelet-config\") pod \"global-pull-secret-syncer-7wl64\" (UID: \"8f732a38-dafb-42d2-acc6-ff8059c65350\") " pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:40:32.607374 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607300 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-etc-openvswitch\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.607374 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607342 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-lib-modules\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.608231 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607397 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffkvq\" (UniqueName: \"kubernetes.io/projected/fa0ec5ee-8ecd-4f22-ae99-cfbe2855395c-kube-api-access-ffkvq\") pod \"node-resolver-qvwd8\" (UID: \"fa0ec5ee-8ecd-4f22-ae99-cfbe2855395c\") " pod="openshift-dns/node-resolver-qvwd8" Aug 04 09:40:32.608231 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607438 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-host-run-netns\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.608231 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607462 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:40:32.608231 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607469 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-run\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.608231 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607492 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d675e416-204d-4e33-958e-ebdc867294d4-tmp\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.608231 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607517 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a9ba7a09-0ff2-4657-8bbd-47b0af96a45a-host\") pod \"node-ca-57rd7\" (UID: \"a9ba7a09-0ff2-4657-8bbd-47b0af96a45a\") " pod="openshift-image-registry/node-ca-57rd7" Aug 04 09:40:32.608231 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:32.607519 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcql5" podUID="6cc7e819-7680-4323-8afa-afa00c9433ce" Aug 04 09:40:32.608231 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607541 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-host-slash\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.608231 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607570 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-run-systemd\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.608231 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607593 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-run-ovn\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.608231 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607618 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Aug 04 09:40:32.608231 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607621 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-host-cni-bin\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.608231 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607744 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-etc-modprobe-d\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.608231 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607811 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-etc-systemd\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.608231 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607847 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d675e416-204d-4e33-958e-ebdc867294d4-etc-tuned\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.608231 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607853 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Aug 04 09:40:32.608231 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607897 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8f732a38-dafb-42d2-acc6-ff8059c65350-dbus\") pod \"global-pull-secret-syncer-7wl64\" (UID: \"8f732a38-dafb-42d2-acc6-ff8059c65350\") " pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:40:32.608231 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607923 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-host-kubelet\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.608843 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607961 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-systemd-units\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.608843 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607970 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-npfrv\"" Aug 04 09:40:32.608843 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.607982 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7858cfca-73db-4ee1-9882-c13444f112a5-env-overrides\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.608843 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.608002 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-etc-sysctl-d\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.608843 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.608024 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x6f8\" (UniqueName: \"kubernetes.io/projected/d675e416-204d-4e33-958e-ebdc867294d4-kube-api-access-7x6f8\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.608843 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.608056 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fa0ec5ee-8ecd-4f22-ae99-cfbe2855395c-hosts-file\") pod \"node-resolver-qvwd8\" (UID: \"fa0ec5ee-8ecd-4f22-ae99-cfbe2855395c\") " pod="openshift-dns/node-resolver-qvwd8" Aug 04 09:40:32.608843 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.608096 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa0ec5ee-8ecd-4f22-ae99-cfbe2855395c-tmp-dir\") pod \"node-resolver-qvwd8\" (UID: \"fa0ec5ee-8ecd-4f22-ae99-cfbe2855395c\") " pod="openshift-dns/node-resolver-qvwd8" Aug 04 09:40:32.608843 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.608120 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f732a38-dafb-42d2-acc6-ff8059c65350-original-pull-secret\") pod \"global-pull-secret-syncer-7wl64\" (UID: \"8f732a38-dafb-42d2-acc6-ff8059c65350\") " pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:40:32.608843 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.608144 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Aug 04 09:40:32.608843 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.608151 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/781f670f-6934-4504-810d-95429f368c7f-agent-certs\") pod \"konnectivity-agent-hgcgb\" (UID: \"781f670f-6934-4504-810d-95429f368c7f\") " pod="kube-system/konnectivity-agent-hgcgb" Aug 04 09:40:32.608843 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.608173 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-node-log\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.608843 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.608195 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Aug 04 09:40:32.608843 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.608203 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-host-run-ovn-kubernetes\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.608843 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.608224 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7858cfca-73db-4ee1-9882-c13444f112a5-ovnkube-config\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.608843 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.608248 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7858cfca-73db-4ee1-9882-c13444f112a5-ovnkube-script-lib\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.608843 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.608277 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a9ba7a09-0ff2-4657-8bbd-47b0af96a45a-serviceca\") pod \"node-ca-57rd7\" (UID: \"a9ba7a09-0ff2-4657-8bbd-47b0af96a45a\") " pod="openshift-image-registry/node-ca-57rd7" Aug 04 09:40:32.608843 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.608304 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-run-openvswitch\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.608843 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.608728 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Aug 04 09:40:32.609870 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.609855 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mbstc" Aug 04 09:40:32.612026 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.612011 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:40:32.612785 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:32.612449 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjdn8" podUID="b7e0b230-f265-4c9b-9d1d-9008457d8aa1" Aug 04 09:40:32.612785 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.612630 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Aug 04 09:40:32.612785 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.612646 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-w25xg\"" Aug 04 09:40:32.612785 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.612659 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Aug 04 09:40:32.612785 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.612752 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Aug 04 09:40:32.615673 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.615653 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdnhg" Aug 04 09:40:32.617889 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.617870 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.618350 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.618323 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Aug 04 09:40:32.618438 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.618385 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Aug 04 09:40:32.618438 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.618404 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vp76h\"" Aug 04 09:40:32.618572 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.618444 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Aug 04 09:40:32.618834 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.618819 2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Aug 04 09:40:32.620418 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.620404 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-dz4lf\"" Aug 04 09:40:32.620580 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.620569 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Aug 04 09:40:32.649873 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.649842 2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-69tv4" Aug 04 09:40:32.658684 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.658660 2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-69tv4" Aug 04 09:40:32.701974 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.701950 2572 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Aug 04 09:40:32.706393 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.706350 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-03 09:35:31 +0000 UTC" deadline="2028-05-06 18:51:35.406799064 +0000 UTC" Aug 04 09:40:32.706393 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.706391 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15393h11m2.70041064s" Aug 04 09:40:32.708578 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.708557 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4df28d6b-d17f-4a28-b25e-b17559454724-device-dir\") pod \"aws-ebs-csi-driver-node-mdnhg\" (UID: \"4df28d6b-d17f-4a28-b25e-b17559454724\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdnhg" Aug 04 09:40:32.708676 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.708582 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4df28d6b-d17f-4a28-b25e-b17559454724-etc-selinux\") pod \"aws-ebs-csi-driver-node-mdnhg\" (UID: \"4df28d6b-d17f-4a28-b25e-b17559454724\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdnhg" Aug 04 09:40:32.708676 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.708601 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-host-run-netns\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.708676 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.708616 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-run\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.708676 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.708667 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/30012dc7-ffb2-44a7-b0fb-b444100f6e31-iptables-alerter-script\") pod \"iptables-alerter-mbstc\" (UID: \"30012dc7-ffb2-44a7-b0fb-b444100f6e31\") " pod="openshift-network-operator/iptables-alerter-mbstc" Aug 04 09:40:32.708791 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.708674 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-run\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.708791 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.708693 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-host-run-netns\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.708791 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.708723 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-run-systemd\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.708791 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.708746 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-etc-modprobe-d\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.708791 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.708783 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-run-systemd\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.709022 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.708823 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-etc-systemd\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.709022 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.708841 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x6f8\" (UniqueName: \"kubernetes.io/projected/d675e416-204d-4e33-958e-ebdc867294d4-kube-api-access-7x6f8\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.709022 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.708861 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-etc-modprobe-d\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.709022 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.708889 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ec918bd2-ac7e-493e-b164-a16ca18c9422-cnibin\") pod \"multus-additional-cni-plugins-2jnfw\" (UID: \"ec918bd2-ac7e-493e-b164-a16ca18c9422\") " pod="openshift-multus/multus-additional-cni-plugins-2jnfw" Aug 04 09:40:32.709022 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.708912 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-etc-systemd\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.709022 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.708916 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q78c\" (UniqueName: \"kubernetes.io/projected/6cc7e819-7680-4323-8afa-afa00c9433ce-kube-api-access-4q78c\") pod \"network-check-target-qcql5\" (UID: \"6cc7e819-7680-4323-8afa-afa00c9433ce\") " pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:40:32.709022 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.708947 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aa641b90-0c09-4fe6-8fa9-6a0a843144db-multus-daemon-config\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.709022 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.708980 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8f732a38-dafb-42d2-acc6-ff8059c65350-dbus\") pod \"global-pull-secret-syncer-7wl64\" (UID: \"8f732a38-dafb-42d2-acc6-ff8059c65350\") " pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:40:32.709022 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709008 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-etc-sysctl-d\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.709457 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709031 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fa0ec5ee-8ecd-4f22-ae99-cfbe2855395c-hosts-file\") pod \"node-resolver-qvwd8\" (UID: \"fa0ec5ee-8ecd-4f22-ae99-cfbe2855395c\") " pod="openshift-dns/node-resolver-qvwd8" Aug 04 09:40:32.709457 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709055 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-host-run-k8s-cni-cncf-io\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.709457 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709068 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8f732a38-dafb-42d2-acc6-ff8059c65350-dbus\") pod \"global-pull-secret-syncer-7wl64\" (UID: \"8f732a38-dafb-42d2-acc6-ff8059c65350\") " pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:40:32.709457 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709082 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7e0b230-f265-4c9b-9d1d-9008457d8aa1-metrics-certs\") pod \"network-metrics-daemon-pjdn8\" (UID: \"b7e0b230-f265-4c9b-9d1d-9008457d8aa1\") " pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:40:32.709457 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709100 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fa0ec5ee-8ecd-4f22-ae99-cfbe2855395c-hosts-file\") pod \"node-resolver-qvwd8\" (UID: \"fa0ec5ee-8ecd-4f22-ae99-cfbe2855395c\") " pod="openshift-dns/node-resolver-qvwd8" Aug 04 09:40:32.709457 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709106 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4df28d6b-d17f-4a28-b25e-b17559454724-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mdnhg\" (UID: \"4df28d6b-d17f-4a28-b25e-b17559454724\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdnhg" Aug 04 09:40:32.709457 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709131 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/781f670f-6934-4504-810d-95429f368c7f-agent-certs\") pod \"konnectivity-agent-hgcgb\" (UID: \"781f670f-6934-4504-810d-95429f368c7f\") " pod="kube-system/konnectivity-agent-hgcgb" Aug 04 09:40:32.709457 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709141 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-etc-sysctl-d\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.709457 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709165 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a9ba7a09-0ff2-4657-8bbd-47b0af96a45a-serviceca\") pod \"node-ca-57rd7\" (UID: \"a9ba7a09-0ff2-4657-8bbd-47b0af96a45a\") " pod="openshift-image-registry/node-ca-57rd7" Aug 04 09:40:32.709457 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709188 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ec918bd2-ac7e-493e-b164-a16ca18c9422-os-release\") pod \"multus-additional-cni-plugins-2jnfw\" (UID: \"ec918bd2-ac7e-493e-b164-a16ca18c9422\") " pod="openshift-multus/multus-additional-cni-plugins-2jnfw" Aug 04 09:40:32.709457 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709204 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ec918bd2-ac7e-493e-b164-a16ca18c9422-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2jnfw\" (UID: \"ec918bd2-ac7e-493e-b164-a16ca18c9422\") " pod="openshift-multus/multus-additional-cni-plugins-2jnfw" Aug 04 09:40:32.709457 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709217 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-cnibin\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.709457 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709233 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5g9p\" (UniqueName: \"kubernetes.io/projected/4df28d6b-d17f-4a28-b25e-b17559454724-kube-api-access-l5g9p\") pod \"aws-ebs-csi-driver-node-mdnhg\" (UID: \"4df28d6b-d17f-4a28-b25e-b17559454724\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdnhg" Aug 04 09:40:32.709457 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709252 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-run-openvswitch\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.709457 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709300 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-etc-sysconfig\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.709457 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709332 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvdcg\" (UniqueName: \"kubernetes.io/projected/a9ba7a09-0ff2-4657-8bbd-47b0af96a45a-kube-api-access-qvdcg\") pod \"node-ca-57rd7\" (UID: \"a9ba7a09-0ff2-4657-8bbd-47b0af96a45a\") " pod="openshift-image-registry/node-ca-57rd7" Aug 04 09:40:32.709457 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709357 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-run-openvswitch\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.710233 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709389 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-etc-sysconfig\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.710233 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709419 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-multus-cni-dir\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.710233 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709453 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6crj\" (UniqueName: \"kubernetes.io/projected/aa641b90-0c09-4fe6-8fa9-6a0a843144db-kube-api-access-d6crj\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.710233 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709479 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-log-socket\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.710233 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709477 2572 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Aug 04 09:40:32.710233 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709525 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-log-socket\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.710233 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709522 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6brcj\" (UniqueName: \"kubernetes.io/projected/7858cfca-73db-4ee1-9882-c13444f112a5-kube-api-access-6brcj\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.710233 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709553 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aa641b90-0c09-4fe6-8fa9-6a0a843144db-cni-binary-copy\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.710233 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709573 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-host-var-lib-kubelet\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.710233 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709601 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-host-run-multus-certs\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.710233 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709617 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-var-lib-openvswitch\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.710233 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709633 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a9ba7a09-0ff2-4657-8bbd-47b0af96a45a-serviceca\") pod \"node-ca-57rd7\" (UID: \"a9ba7a09-0ff2-4657-8bbd-47b0af96a45a\") " pod="openshift-image-registry/node-ca-57rd7" Aug 04 09:40:32.710233 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709662 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-host-cni-netd\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.710233 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709637 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-host-cni-netd\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.710233 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709685 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-var-lib-kubelet\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.710233 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709694 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-var-lib-openvswitch\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.710233 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709705 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/30012dc7-ffb2-44a7-b0fb-b444100f6e31-host-slash\") pod \"iptables-alerter-mbstc\" (UID: \"30012dc7-ffb2-44a7-b0fb-b444100f6e31\") " pod="openshift-network-operator/iptables-alerter-mbstc" Aug 04 09:40:32.710233 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709743 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk2sv\" (UniqueName: \"kubernetes.io/projected/30012dc7-ffb2-44a7-b0fb-b444100f6e31-kube-api-access-qk2sv\") pod \"iptables-alerter-mbstc\" (UID: \"30012dc7-ffb2-44a7-b0fb-b444100f6e31\") " pod="openshift-network-operator/iptables-alerter-mbstc" Aug 04 09:40:32.711129 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709754 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-var-lib-kubelet\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.711129 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709771 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4df28d6b-d17f-4a28-b25e-b17559454724-registration-dir\") pod \"aws-ebs-csi-driver-node-mdnhg\" (UID: \"4df28d6b-d17f-4a28-b25e-b17559454724\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdnhg" Aug 04 09:40:32.711129 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709791 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4df28d6b-d17f-4a28-b25e-b17559454724-sys-fs\") pod \"aws-ebs-csi-driver-node-mdnhg\" (UID: \"4df28d6b-d17f-4a28-b25e-b17559454724\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdnhg" Aug 04 09:40:32.711129 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709831 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ec918bd2-ac7e-493e-b164-a16ca18c9422-system-cni-dir\") pod \"multus-additional-cni-plugins-2jnfw\" (UID: \"ec918bd2-ac7e-493e-b164-a16ca18c9422\") " pod="openshift-multus/multus-additional-cni-plugins-2jnfw" Aug 04 09:40:32.711129 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709846 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ec918bd2-ac7e-493e-b164-a16ca18c9422-cni-binary-copy\") pod \"multus-additional-cni-plugins-2jnfw\" (UID: \"ec918bd2-ac7e-493e-b164-a16ca18c9422\") " pod="openshift-multus/multus-additional-cni-plugins-2jnfw" Aug 04 09:40:32.711129 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709861 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crtb5\" (UniqueName: \"kubernetes.io/projected/ec918bd2-ac7e-493e-b164-a16ca18c9422-kube-api-access-crtb5\") pod \"multus-additional-cni-plugins-2jnfw\" (UID: \"ec918bd2-ac7e-493e-b164-a16ca18c9422\") " pod="openshift-multus/multus-additional-cni-plugins-2jnfw" Aug 04 09:40:32.711129 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709876 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-multus-socket-dir-parent\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.711129 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709896 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-hostroot\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.711129 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709932 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d675e416-204d-4e33-958e-ebdc867294d4-tmp\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.711129 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709954 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a9ba7a09-0ff2-4657-8bbd-47b0af96a45a-host\") pod \"node-ca-57rd7\" (UID: \"a9ba7a09-0ff2-4657-8bbd-47b0af96a45a\") " pod="openshift-image-registry/node-ca-57rd7" Aug 04 09:40:32.711129 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.709975 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-os-release\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.711129 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710022 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a9ba7a09-0ff2-4657-8bbd-47b0af96a45a-host\") pod \"node-ca-57rd7\" (UID: \"a9ba7a09-0ff2-4657-8bbd-47b0af96a45a\") " pod="openshift-image-registry/node-ca-57rd7" Aug 04 09:40:32.711129 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710059 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-host-slash\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.711129 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710086 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-run-ovn\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.711129 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710108 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-host-cni-bin\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.711129 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710131 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d675e416-204d-4e33-958e-ebdc867294d4-etc-tuned\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.711129 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710144 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-host-slash\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.711926 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710155 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa0ec5ee-8ecd-4f22-ae99-cfbe2855395c-tmp-dir\") pod \"node-resolver-qvwd8\" (UID: \"fa0ec5ee-8ecd-4f22-ae99-cfbe2855395c\") " pod="openshift-dns/node-resolver-qvwd8" Aug 04 09:40:32.711926 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710158 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-run-ovn\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.711926 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710169 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-host-cni-bin\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.711926 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710180 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-host-kubelet\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.711926 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710205 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-systemd-units\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.711926 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710228 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7858cfca-73db-4ee1-9882-c13444f112a5-env-overrides\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.711926 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710238 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-host-kubelet\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.711926 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710253 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ec918bd2-ac7e-493e-b164-a16ca18c9422-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2jnfw\" (UID: \"ec918bd2-ac7e-493e-b164-a16ca18c9422\") " pod="openshift-multus/multus-additional-cni-plugins-2jnfw" Aug 04 09:40:32.711926 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710265 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-systemd-units\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.711926 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710282 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-host-var-lib-cni-multus\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.711926 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710324 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4df28d6b-d17f-4a28-b25e-b17559454724-socket-dir\") pod \"aws-ebs-csi-driver-node-mdnhg\" (UID: \"4df28d6b-d17f-4a28-b25e-b17559454724\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdnhg" Aug 04 09:40:32.711926 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710349 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f732a38-dafb-42d2-acc6-ff8059c65350-original-pull-secret\") pod \"global-pull-secret-syncer-7wl64\" (UID: \"8f732a38-dafb-42d2-acc6-ff8059c65350\") " pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:40:32.711926 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710375 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-node-log\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.711926 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710398 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-host-run-ovn-kubernetes\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.711926 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710422 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7858cfca-73db-4ee1-9882-c13444f112a5-ovnkube-config\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.711926 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710446 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7858cfca-73db-4ee1-9882-c13444f112a5-ovnkube-script-lib\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.711926 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710449 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa0ec5ee-8ecd-4f22-ae99-cfbe2855395c-tmp-dir\") pod \"node-resolver-qvwd8\" (UID: \"fa0ec5ee-8ecd-4f22-ae99-cfbe2855395c\") " pod="openshift-dns/node-resolver-qvwd8" Aug 04 09:40:32.712717 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710455 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-node-log\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.712717 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710470 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-multus-conf-dir\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.712717 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710474 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-host-run-ovn-kubernetes\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.712717 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710502 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-sys\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.712717 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710529 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-system-cni-dir\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.712717 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710559 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-sys\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.712717 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710556 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-host-var-lib-cni-bin\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.712717 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710605 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcr75\" (UniqueName: \"kubernetes.io/projected/b7e0b230-f265-4c9b-9d1d-9008457d8aa1-kube-api-access-gcr75\") pod \"network-metrics-daemon-pjdn8\" (UID: \"b7e0b230-f265-4c9b-9d1d-9008457d8aa1\") " pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:40:32.712717 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710636 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.712717 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710662 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7858cfca-73db-4ee1-9882-c13444f112a5-ovn-node-metrics-cert\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.712717 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:32.710679 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 09:40:32.712717 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710687 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-etc-kubernetes\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.712717 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710714 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-etc-kubernetes\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.712717 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710714 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.712717 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:32.710789 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f732a38-dafb-42d2-acc6-ff8059c65350-original-pull-secret podName:8f732a38-dafb-42d2-acc6-ff8059c65350 nodeName:}" failed. No retries permitted until 2026-08-04 09:40:33.210753004 +0000 UTC m=+1.954887416 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f732a38-dafb-42d2-acc6-ff8059c65350-original-pull-secret") pod "global-pull-secret-syncer-7wl64" (UID: "8f732a38-dafb-42d2-acc6-ff8059c65350") : object "kube-system"/"original-pull-secret" not registered Aug 04 09:40:32.712717 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710926 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/781f670f-6934-4504-810d-95429f368c7f-konnectivity-ca\") pod \"konnectivity-agent-hgcgb\" (UID: \"781f670f-6934-4504-810d-95429f368c7f\") " pod="kube-system/konnectivity-agent-hgcgb" Aug 04 09:40:32.712717 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710942 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7858cfca-73db-4ee1-9882-c13444f112a5-env-overrides\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.713439 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710960 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-etc-sysctl-conf\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.713439 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.711004 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-host\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.713439 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.711044 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ec918bd2-ac7e-493e-b164-a16ca18c9422-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2jnfw\" (UID: \"ec918bd2-ac7e-493e-b164-a16ca18c9422\") " pod="openshift-multus/multus-additional-cni-plugins-2jnfw" Aug 04 09:40:32.713439 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.711065 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-host\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.713439 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.711075 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7858cfca-73db-4ee1-9882-c13444f112a5-ovnkube-script-lib\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.713439 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.710831 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-etc-kubernetes\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.713439 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.711075 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8f732a38-dafb-42d2-acc6-ff8059c65350-kubelet-config\") pod \"global-pull-secret-syncer-7wl64\" (UID: \"8f732a38-dafb-42d2-acc6-ff8059c65350\") " pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:40:32.713439 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.711145 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8f732a38-dafb-42d2-acc6-ff8059c65350-kubelet-config\") pod \"global-pull-secret-syncer-7wl64\" (UID: \"8f732a38-dafb-42d2-acc6-ff8059c65350\") " pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:40:32.713439 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.711163 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-etc-openvswitch\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.713439 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.711195 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-etc-sysctl-conf\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.713439 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.711201 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7858cfca-73db-4ee1-9882-c13444f112a5-etc-openvswitch\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.713439 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.711197 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-lib-modules\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.713439 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.711252 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffkvq\" (UniqueName: \"kubernetes.io/projected/fa0ec5ee-8ecd-4f22-ae99-cfbe2855395c-kube-api-access-ffkvq\") pod \"node-resolver-qvwd8\" (UID: \"fa0ec5ee-8ecd-4f22-ae99-cfbe2855395c\") " pod="openshift-dns/node-resolver-qvwd8" Aug 04 09:40:32.713439 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.711279 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-host-run-netns\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.713439 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.711284 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d675e416-204d-4e33-958e-ebdc867294d4-lib-modules\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.713439 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.711413 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/781f670f-6934-4504-810d-95429f368c7f-konnectivity-ca\") pod \"konnectivity-agent-hgcgb\" (UID: \"781f670f-6934-4504-810d-95429f368c7f\") " pod="kube-system/konnectivity-agent-hgcgb" Aug 04 09:40:32.713439 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.711557 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7858cfca-73db-4ee1-9882-c13444f112a5-ovnkube-config\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.713937 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.713071 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d675e416-204d-4e33-958e-ebdc867294d4-tmp\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.713937 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.713212 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7858cfca-73db-4ee1-9882-c13444f112a5-ovn-node-metrics-cert\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.713937 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.713276 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d675e416-204d-4e33-958e-ebdc867294d4-etc-tuned\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.713937 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.713314 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/781f670f-6934-4504-810d-95429f368c7f-agent-certs\") pod \"konnectivity-agent-hgcgb\" (UID: \"781f670f-6934-4504-810d-95429f368c7f\") " pod="kube-system/konnectivity-agent-hgcgb" Aug 04 09:40:32.717987 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.717961 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x6f8\" (UniqueName: \"kubernetes.io/projected/d675e416-204d-4e33-958e-ebdc867294d4-kube-api-access-7x6f8\") pod \"tuned-6n7bh\" (UID: \"d675e416-204d-4e33-958e-ebdc867294d4\") " pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.719282 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.719251 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvdcg\" (UniqueName: \"kubernetes.io/projected/a9ba7a09-0ff2-4657-8bbd-47b0af96a45a-kube-api-access-qvdcg\") pod \"node-ca-57rd7\" (UID: \"a9ba7a09-0ff2-4657-8bbd-47b0af96a45a\") " pod="openshift-image-registry/node-ca-57rd7" Aug 04 09:40:32.720348 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.720327 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6brcj\" (UniqueName: \"kubernetes.io/projected/7858cfca-73db-4ee1-9882-c13444f112a5-kube-api-access-6brcj\") pod \"ovnkube-node-8sdsr\" (UID: \"7858cfca-73db-4ee1-9882-c13444f112a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.720498 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.720483 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffkvq\" (UniqueName: \"kubernetes.io/projected/fa0ec5ee-8ecd-4f22-ae99-cfbe2855395c-kube-api-access-ffkvq\") pod \"node-resolver-qvwd8\" (UID: \"fa0ec5ee-8ecd-4f22-ae99-cfbe2855395c\") " pod="openshift-dns/node-resolver-qvwd8" Aug 04 09:40:32.811710 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.811621 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-multus-conf-dir\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.811710 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.811653 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-system-cni-dir\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.811710 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.811668 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-host-var-lib-cni-bin\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.811710 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.811683 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcr75\" (UniqueName: \"kubernetes.io/projected/b7e0b230-f265-4c9b-9d1d-9008457d8aa1-kube-api-access-gcr75\") pod \"network-metrics-daemon-pjdn8\" (UID: \"b7e0b230-f265-4c9b-9d1d-9008457d8aa1\") " pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:40:32.811710 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.811701 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-etc-kubernetes\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.812086 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.811722 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ec918bd2-ac7e-493e-b164-a16ca18c9422-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2jnfw\" (UID: \"ec918bd2-ac7e-493e-b164-a16ca18c9422\") " pod="openshift-multus/multus-additional-cni-plugins-2jnfw" Aug 04 09:40:32.812086 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.811748 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-host-run-netns\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.812086 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.811755 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-system-cni-dir\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.812086 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.811755 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-host-var-lib-cni-bin\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.812086 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.811760 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-multus-conf-dir\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.812086 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.811812 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-host-run-netns\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.812086 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.811814 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-etc-kubernetes\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.812086 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.811850 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4df28d6b-d17f-4a28-b25e-b17559454724-device-dir\") pod \"aws-ebs-csi-driver-node-mdnhg\" (UID: \"4df28d6b-d17f-4a28-b25e-b17559454724\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdnhg" Aug 04 09:40:32.812086 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.811889 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4df28d6b-d17f-4a28-b25e-b17559454724-etc-selinux\") pod \"aws-ebs-csi-driver-node-mdnhg\" (UID: \"4df28d6b-d17f-4a28-b25e-b17559454724\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdnhg" Aug 04 09:40:32.812086 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.811917 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/30012dc7-ffb2-44a7-b0fb-b444100f6e31-iptables-alerter-script\") pod \"iptables-alerter-mbstc\" (UID: \"30012dc7-ffb2-44a7-b0fb-b444100f6e31\") " pod="openshift-network-operator/iptables-alerter-mbstc" Aug 04 09:40:32.812086 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.811929 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4df28d6b-d17f-4a28-b25e-b17559454724-device-dir\") pod \"aws-ebs-csi-driver-node-mdnhg\" (UID: \"4df28d6b-d17f-4a28-b25e-b17559454724\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdnhg" Aug 04 09:40:32.812086 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.811949 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ec918bd2-ac7e-493e-b164-a16ca18c9422-cnibin\") pod \"multus-additional-cni-plugins-2jnfw\" (UID: \"ec918bd2-ac7e-493e-b164-a16ca18c9422\") " pod="openshift-multus/multus-additional-cni-plugins-2jnfw" Aug 04 09:40:32.812086 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.811982 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q78c\" (UniqueName: \"kubernetes.io/projected/6cc7e819-7680-4323-8afa-afa00c9433ce-kube-api-access-4q78c\") pod \"network-check-target-qcql5\" (UID: \"6cc7e819-7680-4323-8afa-afa00c9433ce\") " pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:40:32.812086 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.811983 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ec918bd2-ac7e-493e-b164-a16ca18c9422-cnibin\") pod \"multus-additional-cni-plugins-2jnfw\" (UID: \"ec918bd2-ac7e-493e-b164-a16ca18c9422\") " pod="openshift-multus/multus-additional-cni-plugins-2jnfw" Aug 04 09:40:32.812086 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812025 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aa641b90-0c09-4fe6-8fa9-6a0a843144db-multus-daemon-config\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.812086 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812050 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-host-run-k8s-cni-cncf-io\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.812086 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812053 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4df28d6b-d17f-4a28-b25e-b17559454724-etc-selinux\") pod \"aws-ebs-csi-driver-node-mdnhg\" (UID: \"4df28d6b-d17f-4a28-b25e-b17559454724\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdnhg" Aug 04 09:40:32.812832 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812070 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7e0b230-f265-4c9b-9d1d-9008457d8aa1-metrics-certs\") pod \"network-metrics-daemon-pjdn8\" (UID: \"b7e0b230-f265-4c9b-9d1d-9008457d8aa1\") " pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:40:32.812832 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812092 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4df28d6b-d17f-4a28-b25e-b17559454724-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mdnhg\" (UID: \"4df28d6b-d17f-4a28-b25e-b17559454724\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdnhg" Aug 04 09:40:32.812832 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812104 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-host-run-k8s-cni-cncf-io\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.812832 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812118 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ec918bd2-ac7e-493e-b164-a16ca18c9422-os-release\") pod \"multus-additional-cni-plugins-2jnfw\" (UID: \"ec918bd2-ac7e-493e-b164-a16ca18c9422\") " pod="openshift-multus/multus-additional-cni-plugins-2jnfw" Aug 04 09:40:32.812832 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812143 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ec918bd2-ac7e-493e-b164-a16ca18c9422-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2jnfw\" (UID: \"ec918bd2-ac7e-493e-b164-a16ca18c9422\") " pod="openshift-multus/multus-additional-cni-plugins-2jnfw" Aug 04 09:40:32.812832 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812158 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4df28d6b-d17f-4a28-b25e-b17559454724-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mdnhg\" (UID: \"4df28d6b-d17f-4a28-b25e-b17559454724\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdnhg" Aug 04 09:40:32.812832 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812166 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-cnibin\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.812832 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812191 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5g9p\" (UniqueName: \"kubernetes.io/projected/4df28d6b-d17f-4a28-b25e-b17559454724-kube-api-access-l5g9p\") pod \"aws-ebs-csi-driver-node-mdnhg\" (UID: \"4df28d6b-d17f-4a28-b25e-b17559454724\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdnhg" Aug 04 09:40:32.812832 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:32.812196 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:40:32.812832 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812218 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-multus-cni-dir\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.812832 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812235 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ec918bd2-ac7e-493e-b164-a16ca18c9422-os-release\") pod \"multus-additional-cni-plugins-2jnfw\" (UID: \"ec918bd2-ac7e-493e-b164-a16ca18c9422\") " pod="openshift-multus/multus-additional-cni-plugins-2jnfw" Aug 04 09:40:32.812832 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812240 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6crj\" (UniqueName: \"kubernetes.io/projected/aa641b90-0c09-4fe6-8fa9-6a0a843144db-kube-api-access-d6crj\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.812832 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812268 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aa641b90-0c09-4fe6-8fa9-6a0a843144db-cni-binary-copy\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.812832 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812291 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-host-var-lib-kubelet\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.812832 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812312 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-host-run-multus-certs\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.812832 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812328 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ec918bd2-ac7e-493e-b164-a16ca18c9422-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2jnfw\" (UID: \"ec918bd2-ac7e-493e-b164-a16ca18c9422\") " pod="openshift-multus/multus-additional-cni-plugins-2jnfw" Aug 04 09:40:32.812832 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812339 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/30012dc7-ffb2-44a7-b0fb-b444100f6e31-host-slash\") pod \"iptables-alerter-mbstc\" (UID: \"30012dc7-ffb2-44a7-b0fb-b444100f6e31\") " pod="openshift-network-operator/iptables-alerter-mbstc" Aug 04 09:40:32.813309 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:32.812381 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7e0b230-f265-4c9b-9d1d-9008457d8aa1-metrics-certs podName:b7e0b230-f265-4c9b-9d1d-9008457d8aa1 nodeName:}" failed. No retries permitted until 2026-08-04 09:40:33.312363238 +0000 UTC m=+2.056497636 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7e0b230-f265-4c9b-9d1d-9008457d8aa1-metrics-certs") pod "network-metrics-daemon-pjdn8" (UID: "b7e0b230-f265-4c9b-9d1d-9008457d8aa1") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:40:32.813309 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812397 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk2sv\" (UniqueName: \"kubernetes.io/projected/30012dc7-ffb2-44a7-b0fb-b444100f6e31-kube-api-access-qk2sv\") pod \"iptables-alerter-mbstc\" (UID: \"30012dc7-ffb2-44a7-b0fb-b444100f6e31\") " pod="openshift-network-operator/iptables-alerter-mbstc" Aug 04 09:40:32.813309 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812400 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-multus-cni-dir\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.813309 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812423 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4df28d6b-d17f-4a28-b25e-b17559454724-registration-dir\") pod \"aws-ebs-csi-driver-node-mdnhg\" (UID: \"4df28d6b-d17f-4a28-b25e-b17559454724\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdnhg" Aug 04 09:40:32.813309 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812427 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-cnibin\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.813309 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812432 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ec918bd2-ac7e-493e-b164-a16ca18c9422-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2jnfw\" (UID: \"ec918bd2-ac7e-493e-b164-a16ca18c9422\") " pod="openshift-multus/multus-additional-cni-plugins-2jnfw" Aug 04 09:40:32.813309 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812458 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/30012dc7-ffb2-44a7-b0fb-b444100f6e31-iptables-alerter-script\") pod \"iptables-alerter-mbstc\" (UID: \"30012dc7-ffb2-44a7-b0fb-b444100f6e31\") " pod="openshift-network-operator/iptables-alerter-mbstc" Aug 04 09:40:32.813309 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812459 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4df28d6b-d17f-4a28-b25e-b17559454724-sys-fs\") pod \"aws-ebs-csi-driver-node-mdnhg\" (UID: \"4df28d6b-d17f-4a28-b25e-b17559454724\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdnhg" Aug 04 09:40:32.813309 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812499 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4df28d6b-d17f-4a28-b25e-b17559454724-sys-fs\") pod \"aws-ebs-csi-driver-node-mdnhg\" (UID: \"4df28d6b-d17f-4a28-b25e-b17559454724\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdnhg" Aug 04 09:40:32.813309 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812505 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ec918bd2-ac7e-493e-b164-a16ca18c9422-system-cni-dir\") pod \"multus-additional-cni-plugins-2jnfw\" (UID: \"ec918bd2-ac7e-493e-b164-a16ca18c9422\") " pod="openshift-multus/multus-additional-cni-plugins-2jnfw" Aug 04 09:40:32.813309 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812508 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-host-var-lib-kubelet\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.813309 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812536 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/30012dc7-ffb2-44a7-b0fb-b444100f6e31-host-slash\") pod \"iptables-alerter-mbstc\" (UID: \"30012dc7-ffb2-44a7-b0fb-b444100f6e31\") " pod="openshift-network-operator/iptables-alerter-mbstc" Aug 04 09:40:32.813309 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812547 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ec918bd2-ac7e-493e-b164-a16ca18c9422-system-cni-dir\") pod \"multus-additional-cni-plugins-2jnfw\" (UID: \"ec918bd2-ac7e-493e-b164-a16ca18c9422\") " pod="openshift-multus/multus-additional-cni-plugins-2jnfw" Aug 04 09:40:32.813309 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812557 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-host-run-multus-certs\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.813309 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812574 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ec918bd2-ac7e-493e-b164-a16ca18c9422-cni-binary-copy\") pod \"multus-additional-cni-plugins-2jnfw\" (UID: \"ec918bd2-ac7e-493e-b164-a16ca18c9422\") " pod="openshift-multus/multus-additional-cni-plugins-2jnfw" Aug 04 09:40:32.813309 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812582 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4df28d6b-d17f-4a28-b25e-b17559454724-registration-dir\") pod \"aws-ebs-csi-driver-node-mdnhg\" (UID: \"4df28d6b-d17f-4a28-b25e-b17559454724\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdnhg" Aug 04 09:40:32.813744 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812610 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crtb5\" (UniqueName: \"kubernetes.io/projected/ec918bd2-ac7e-493e-b164-a16ca18c9422-kube-api-access-crtb5\") pod \"multus-additional-cni-plugins-2jnfw\" (UID: \"ec918bd2-ac7e-493e-b164-a16ca18c9422\") " pod="openshift-multus/multus-additional-cni-plugins-2jnfw" Aug 04 09:40:32.813744 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812618 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aa641b90-0c09-4fe6-8fa9-6a0a843144db-multus-daemon-config\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.813744 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812635 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-multus-socket-dir-parent\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.813744 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812688 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-hostroot\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.813744 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812714 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-os-release\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.813744 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812749 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ec918bd2-ac7e-493e-b164-a16ca18c9422-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2jnfw\" (UID: \"ec918bd2-ac7e-493e-b164-a16ca18c9422\") " pod="openshift-multus/multus-additional-cni-plugins-2jnfw" Aug 04 09:40:32.813744 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812752 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-hostroot\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.813744 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812762 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-multus-socket-dir-parent\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.813744 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812776 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-host-var-lib-cni-multus\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.813744 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812836 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-os-release\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.813744 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812859 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4df28d6b-d17f-4a28-b25e-b17559454724-socket-dir\") pod \"aws-ebs-csi-driver-node-mdnhg\" (UID: \"4df28d6b-d17f-4a28-b25e-b17559454724\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdnhg" Aug 04 09:40:32.813744 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812927 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aa641b90-0c09-4fe6-8fa9-6a0a843144db-host-var-lib-cni-multus\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.813744 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812951 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aa641b90-0c09-4fe6-8fa9-6a0a843144db-cni-binary-copy\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.813744 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.812987 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4df28d6b-d17f-4a28-b25e-b17559454724-socket-dir\") pod \"aws-ebs-csi-driver-node-mdnhg\" (UID: \"4df28d6b-d17f-4a28-b25e-b17559454724\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdnhg" Aug 04 09:40:32.813744 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.813036 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ec918bd2-ac7e-493e-b164-a16ca18c9422-cni-binary-copy\") pod \"multus-additional-cni-plugins-2jnfw\" (UID: \"ec918bd2-ac7e-493e-b164-a16ca18c9422\") " pod="openshift-multus/multus-additional-cni-plugins-2jnfw" Aug 04 09:40:32.813744 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.813176 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ec918bd2-ac7e-493e-b164-a16ca18c9422-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2jnfw\" (UID: \"ec918bd2-ac7e-493e-b164-a16ca18c9422\") " pod="openshift-multus/multus-additional-cni-plugins-2jnfw" Aug 04 09:40:32.821506 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:32.821486 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 09:40:32.821566 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:32.821511 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 09:40:32.821566 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:32.821523 2572 projected.go:196] Error preparing data for projected volume kube-api-access-4q78c for pod openshift-network-diagnostics/network-check-target-qcql5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:40:32.821623 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:32.821580 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6cc7e819-7680-4323-8afa-afa00c9433ce-kube-api-access-4q78c podName:6cc7e819-7680-4323-8afa-afa00c9433ce nodeName:}" failed. No retries permitted until 2026-08-04 09:40:33.321565845 +0000 UTC m=+2.065700245 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4q78c" (UniqueName: "kubernetes.io/projected/6cc7e819-7680-4323-8afa-afa00c9433ce-kube-api-access-4q78c") pod "network-check-target-qcql5" (UID: "6cc7e819-7680-4323-8afa-afa00c9433ce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:40:32.824070 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.824045 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcr75\" (UniqueName: \"kubernetes.io/projected/b7e0b230-f265-4c9b-9d1d-9008457d8aa1-kube-api-access-gcr75\") pod \"network-metrics-daemon-pjdn8\" (UID: \"b7e0b230-f265-4c9b-9d1d-9008457d8aa1\") " pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:40:32.824426 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.824401 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crtb5\" (UniqueName: \"kubernetes.io/projected/ec918bd2-ac7e-493e-b164-a16ca18c9422-kube-api-access-crtb5\") pod \"multus-additional-cni-plugins-2jnfw\" (UID: \"ec918bd2-ac7e-493e-b164-a16ca18c9422\") " pod="openshift-multus/multus-additional-cni-plugins-2jnfw" Aug 04 09:40:32.825161 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.825137 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5g9p\" (UniqueName: \"kubernetes.io/projected/4df28d6b-d17f-4a28-b25e-b17559454724-kube-api-access-l5g9p\") pod \"aws-ebs-csi-driver-node-mdnhg\" (UID: \"4df28d6b-d17f-4a28-b25e-b17559454724\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdnhg" Aug 04 09:40:32.825263 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.825249 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6crj\" (UniqueName: \"kubernetes.io/projected/aa641b90-0c09-4fe6-8fa9-6a0a843144db-kube-api-access-d6crj\") pod \"multus-94bwj\" (UID: \"aa641b90-0c09-4fe6-8fa9-6a0a843144db\") " pod="openshift-multus/multus-94bwj" Aug 04 09:40:32.826190 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.826173 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk2sv\" (UniqueName: \"kubernetes.io/projected/30012dc7-ffb2-44a7-b0fb-b444100f6e31-kube-api-access-qk2sv\") pod \"iptables-alerter-mbstc\" (UID: \"30012dc7-ffb2-44a7-b0fb-b444100f6e31\") " pod="openshift-network-operator/iptables-alerter-mbstc" Aug 04 09:40:32.917887 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.917854 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-57rd7" Aug 04 09:40:32.935100 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.935072 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qvwd8" Aug 04 09:40:32.955739 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.955709 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hgcgb" Aug 04 09:40:32.973634 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.973609 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:32.989365 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.989335 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" Aug 04 09:40:32.993965 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:32.993942 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2jnfw" Aug 04 09:40:33.028969 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:33.028938 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mbstc" Aug 04 09:40:33.044134 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:33.044105 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdnhg" Aug 04 09:40:33.049088 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:33.049066 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-94bwj" Aug 04 09:40:33.163587 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:33.163507 2572 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 09:40:33.180209 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:33.180173 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0c99023fafad27b36808bb52bb184dd.slice/crio-290dc3686c5d6bb1e07c989efaa7a44e9bc76a945b5abbb1552f19b55019bf0c WatchSource:0}: Error finding container 290dc3686c5d6bb1e07c989efaa7a44e9bc76a945b5abbb1552f19b55019bf0c: Status 404 returned error can't find the container with id 290dc3686c5d6bb1e07c989efaa7a44e9bc76a945b5abbb1552f19b55019bf0c Aug 04 09:40:33.180845 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:33.180823 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88730f4d5c4cac33f5b85009ff17427c.slice/crio-562eb547c1b399fd286604c6696de6d0c24149178085091a36abac705948a5d7 WatchSource:0}: Error finding container 562eb547c1b399fd286604c6696de6d0c24149178085091a36abac705948a5d7: Status 404 returned error can't find the container with id 562eb547c1b399fd286604c6696de6d0c24149178085091a36abac705948a5d7 Aug 04 09:40:33.184784 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:33.184762 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 09:40:33.216281 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:33.216241 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f732a38-dafb-42d2-acc6-ff8059c65350-original-pull-secret\") pod \"global-pull-secret-syncer-7wl64\" (UID: \"8f732a38-dafb-42d2-acc6-ff8059c65350\") " pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:40:33.216460 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:33.216351 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 09:40:33.216460 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:33.216425 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f732a38-dafb-42d2-acc6-ff8059c65350-original-pull-secret podName:8f732a38-dafb-42d2-acc6-ff8059c65350 nodeName:}" failed. No retries permitted until 2026-08-04 09:40:34.216411342 +0000 UTC m=+2.960545739 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f732a38-dafb-42d2-acc6-ff8059c65350-original-pull-secret") pod "global-pull-secret-syncer-7wl64" (UID: "8f732a38-dafb-42d2-acc6-ff8059c65350") : object "kube-system"/"original-pull-secret" not registered Aug 04 09:40:33.316814 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:33.316770 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7e0b230-f265-4c9b-9d1d-9008457d8aa1-metrics-certs\") pod \"network-metrics-daemon-pjdn8\" (UID: \"b7e0b230-f265-4c9b-9d1d-9008457d8aa1\") " pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:40:33.316987 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:33.316924 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:40:33.317060 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:33.317010 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7e0b230-f265-4c9b-9d1d-9008457d8aa1-metrics-certs podName:b7e0b230-f265-4c9b-9d1d-9008457d8aa1 nodeName:}" failed. No retries permitted until 2026-08-04 09:40:34.316988907 +0000 UTC m=+3.061123321 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7e0b230-f265-4c9b-9d1d-9008457d8aa1-metrics-certs") pod "network-metrics-daemon-pjdn8" (UID: "b7e0b230-f265-4c9b-9d1d-9008457d8aa1") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:40:33.350809 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:33.350760 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod781f670f_6934_4504_810d_95429f368c7f.slice/crio-2dbfeb2ca2a5aa0b1ae93122705fa7d9e87db223cb714fe4da839b47d6e6d067 WatchSource:0}: Error finding container 2dbfeb2ca2a5aa0b1ae93122705fa7d9e87db223cb714fe4da839b47d6e6d067: Status 404 returned error can't find the container with id 2dbfeb2ca2a5aa0b1ae93122705fa7d9e87db223cb714fe4da839b47d6e6d067 Aug 04 09:40:33.360310 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:33.360277 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd675e416_204d_4e33_958e_ebdc867294d4.slice/crio-65abd06de379b288095f3cce6a0cc78762c2bf1501e860be4fd576fadde538cf WatchSource:0}: Error finding container 65abd06de379b288095f3cce6a0cc78762c2bf1501e860be4fd576fadde538cf: Status 404 returned error can't find the container with id 65abd06de379b288095f3cce6a0cc78762c2bf1501e860be4fd576fadde538cf Aug 04 09:40:33.381124 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:33.381099 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30012dc7_ffb2_44a7_b0fb_b444100f6e31.slice/crio-df3c4da5d96e838595d680caaa40130804c10f92de19b40e46dc1eefd7648c1c WatchSource:0}: Error finding container df3c4da5d96e838595d680caaa40130804c10f92de19b40e46dc1eefd7648c1c: Status 404 returned error can't find the container with id df3c4da5d96e838595d680caaa40130804c10f92de19b40e46dc1eefd7648c1c Aug 04 09:40:33.417393 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:33.417365 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q78c\" (UniqueName: \"kubernetes.io/projected/6cc7e819-7680-4323-8afa-afa00c9433ce-kube-api-access-4q78c\") pod \"network-check-target-qcql5\" (UID: \"6cc7e819-7680-4323-8afa-afa00c9433ce\") " pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:40:33.417552 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:33.417534 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 09:40:33.417609 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:33.417558 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 09:40:33.417609 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:33.417570 2572 projected.go:196] Error preparing data for projected volume kube-api-access-4q78c for pod openshift-network-diagnostics/network-check-target-qcql5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:40:33.417704 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:33.417625 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6cc7e819-7680-4323-8afa-afa00c9433ce-kube-api-access-4q78c podName:6cc7e819-7680-4323-8afa-afa00c9433ce nodeName:}" failed. No retries permitted until 2026-08-04 09:40:34.417610799 +0000 UTC m=+3.161745197 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4q78c" (UniqueName: "kubernetes.io/projected/6cc7e819-7680-4323-8afa-afa00c9433ce-kube-api-access-4q78c") pod "network-check-target-qcql5" (UID: "6cc7e819-7680-4323-8afa-afa00c9433ce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:40:33.418472 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:33.418449 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec918bd2_ac7e_493e_b164_a16ca18c9422.slice/crio-8dce66daa71fb380177e9ec7095755e469539be3c0e9e3cc532577e5db801707 WatchSource:0}: Error finding container 8dce66daa71fb380177e9ec7095755e469539be3c0e9e3cc532577e5db801707: Status 404 returned error can't find the container with id 8dce66daa71fb380177e9ec7095755e469539be3c0e9e3cc532577e5db801707 Aug 04 09:40:33.437502 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:33.437478 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa0ec5ee_8ecd_4f22_ae99_cfbe2855395c.slice/crio-c5f6a6bbe0f6d00be3929c28ad53e2d91f5f797f2fd31054e1e46ea7ea412c77 WatchSource:0}: Error finding container c5f6a6bbe0f6d00be3929c28ad53e2d91f5f797f2fd31054e1e46ea7ea412c77: Status 404 returned error can't find the container with id c5f6a6bbe0f6d00be3929c28ad53e2d91f5f797f2fd31054e1e46ea7ea412c77 Aug 04 09:40:33.443743 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:33.443724 2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 09:40:33.471686 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:33.471653 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7858cfca_73db_4ee1_9882_c13444f112a5.slice/crio-64dec7275127775a95f7c35732b322f412ab987a4fb7709e457dac483ebcdf8f WatchSource:0}: Error finding container 64dec7275127775a95f7c35732b322f412ab987a4fb7709e457dac483ebcdf8f: Status 404 returned error can't find the container with id 64dec7275127775a95f7c35732b322f412ab987a4fb7709e457dac483ebcdf8f Aug 04 09:40:33.661728 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:33.661652 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 09:35:32 +0000 UTC" deadline="2028-03-19 06:41:11.825013382 +0000 UTC" Aug 04 09:40:33.661728 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:33.661717 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14229h0m38.163298832s" Aug 04 09:40:33.729583 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:33.729539 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa641b90_0c09_4fe6_8fa9_6a0a843144db.slice/crio-e3ed29975a8a6a63ade15166b93af12d0d27beec11d10abb428bf7ba3fc8ca01 WatchSource:0}: Error finding container e3ed29975a8a6a63ade15166b93af12d0d27beec11d10abb428bf7ba3fc8ca01: Status 404 returned error can't find the container with id e3ed29975a8a6a63ade15166b93af12d0d27beec11d10abb428bf7ba3fc8ca01 Aug 04 09:40:33.773117 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:33.773084 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:40:33.773311 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:33.773202 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjdn8" podUID="b7e0b230-f265-4c9b-9d1d-9008457d8aa1" Aug 04 09:40:33.777323 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:33.777257 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mbstc" event={"ID":"30012dc7-ffb2-44a7-b0fb-b444100f6e31","Type":"ContainerStarted","Data":"df3c4da5d96e838595d680caaa40130804c10f92de19b40e46dc1eefd7648c1c"} Aug 04 09:40:33.778707 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:33.778663 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" event={"ID":"d675e416-204d-4e33-958e-ebdc867294d4","Type":"ContainerStarted","Data":"65abd06de379b288095f3cce6a0cc78762c2bf1501e860be4fd576fadde538cf"} Aug 04 09:40:33.780423 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:33.780390 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-94bwj" event={"ID":"aa641b90-0c09-4fe6-8fa9-6a0a843144db","Type":"ContainerStarted","Data":"e3ed29975a8a6a63ade15166b93af12d0d27beec11d10abb428bf7ba3fc8ca01"} Aug 04 09:40:33.781807 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:33.781761 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" event={"ID":"7858cfca-73db-4ee1-9882-c13444f112a5","Type":"ContainerStarted","Data":"64dec7275127775a95f7c35732b322f412ab987a4fb7709e457dac483ebcdf8f"} Aug 04 09:40:33.783273 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:33.783242 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qvwd8" event={"ID":"fa0ec5ee-8ecd-4f22-ae99-cfbe2855395c","Type":"ContainerStarted","Data":"c5f6a6bbe0f6d00be3929c28ad53e2d91f5f797f2fd31054e1e46ea7ea412c77"} Aug 04 09:40:33.784609 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:33.784577 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2jnfw" event={"ID":"ec918bd2-ac7e-493e-b164-a16ca18c9422","Type":"ContainerStarted","Data":"8dce66daa71fb380177e9ec7095755e469539be3c0e9e3cc532577e5db801707"} Aug 04 09:40:33.785995 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:33.785970 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hgcgb" event={"ID":"781f670f-6934-4504-810d-95429f368c7f","Type":"ContainerStarted","Data":"2dbfeb2ca2a5aa0b1ae93122705fa7d9e87db223cb714fe4da839b47d6e6d067"} Aug 04 09:40:33.787555 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:33.787529 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-241.ec2.internal" event={"ID":"f0c99023fafad27b36808bb52bb184dd","Type":"ContainerStarted","Data":"290dc3686c5d6bb1e07c989efaa7a44e9bc76a945b5abbb1552f19b55019bf0c"} Aug 04 09:40:33.789197 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:33.789168 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-241.ec2.internal" event={"ID":"88730f4d5c4cac33f5b85009ff17427c","Type":"ContainerStarted","Data":"562eb547c1b399fd286604c6696de6d0c24149178085091a36abac705948a5d7"} Aug 04 09:40:33.790889 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:33.790867 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-57rd7" event={"ID":"a9ba7a09-0ff2-4657-8bbd-47b0af96a45a","Type":"ContainerStarted","Data":"ff8b662152cedc475fb977cfe790d7dbfc8eb41c67be2227a2485e9347f6549e"} Aug 04 09:40:34.046883 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:40:34.046859 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4df28d6b_d17f_4a28_b25e_b17559454724.slice/crio-dcbe5d7bd093d9c6494814b1b07a0da038924d727c7e7c2ab9ca4551bbf13f47 WatchSource:0}: Error finding container dcbe5d7bd093d9c6494814b1b07a0da038924d727c7e7c2ab9ca4551bbf13f47: Status 404 returned error can't find the container with id dcbe5d7bd093d9c6494814b1b07a0da038924d727c7e7c2ab9ca4551bbf13f47 Aug 04 09:40:34.225659 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:34.225620 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f732a38-dafb-42d2-acc6-ff8059c65350-original-pull-secret\") pod \"global-pull-secret-syncer-7wl64\" (UID: \"8f732a38-dafb-42d2-acc6-ff8059c65350\") " pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:40:34.226291 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:34.226266 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 09:40:34.226415 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:34.226357 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f732a38-dafb-42d2-acc6-ff8059c65350-original-pull-secret podName:8f732a38-dafb-42d2-acc6-ff8059c65350 nodeName:}" failed. No retries permitted until 2026-08-04 09:40:36.226336644 +0000 UTC m=+4.970471063 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f732a38-dafb-42d2-acc6-ff8059c65350-original-pull-secret") pod "global-pull-secret-syncer-7wl64" (UID: "8f732a38-dafb-42d2-acc6-ff8059c65350") : object "kube-system"/"original-pull-secret" not registered Aug 04 09:40:34.326937 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:34.326739 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7e0b230-f265-4c9b-9d1d-9008457d8aa1-metrics-certs\") pod \"network-metrics-daemon-pjdn8\" (UID: \"b7e0b230-f265-4c9b-9d1d-9008457d8aa1\") " pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:40:34.326937 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:34.326914 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:40:34.327212 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:34.326978 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7e0b230-f265-4c9b-9d1d-9008457d8aa1-metrics-certs podName:b7e0b230-f265-4c9b-9d1d-9008457d8aa1 nodeName:}" failed. No retries permitted until 2026-08-04 09:40:36.326960234 +0000 UTC m=+5.071094646 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7e0b230-f265-4c9b-9d1d-9008457d8aa1-metrics-certs") pod "network-metrics-daemon-pjdn8" (UID: "b7e0b230-f265-4c9b-9d1d-9008457d8aa1") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:40:34.427929 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:34.427889 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q78c\" (UniqueName: \"kubernetes.io/projected/6cc7e819-7680-4323-8afa-afa00c9433ce-kube-api-access-4q78c\") pod \"network-check-target-qcql5\" (UID: \"6cc7e819-7680-4323-8afa-afa00c9433ce\") " pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:40:34.428126 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:34.428091 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 09:40:34.428217 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:34.428118 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 09:40:34.428217 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:34.428167 2572 projected.go:196] Error preparing data for projected volume kube-api-access-4q78c for pod openshift-network-diagnostics/network-check-target-qcql5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:40:34.428319 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:34.428231 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6cc7e819-7680-4323-8afa-afa00c9433ce-kube-api-access-4q78c podName:6cc7e819-7680-4323-8afa-afa00c9433ce nodeName:}" failed. No retries permitted until 2026-08-04 09:40:36.428211682 +0000 UTC m=+5.172346097 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4q78c" (UniqueName: "kubernetes.io/projected/6cc7e819-7680-4323-8afa-afa00c9433ce-kube-api-access-4q78c") pod "network-check-target-qcql5" (UID: "6cc7e819-7680-4323-8afa-afa00c9433ce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:40:34.662128 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:34.662009 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 09:35:32 +0000 UTC" deadline="2028-03-10 22:01:26.89249239 +0000 UTC" Aug 04 09:40:34.662128 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:34.662049 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14028h20m52.230449338s" Aug 04 09:40:34.772754 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:34.772718 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:40:34.772951 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:34.772763 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:40:34.772951 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:34.772872 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wl64" podUID="8f732a38-dafb-42d2-acc6-ff8059c65350" Aug 04 09:40:34.773310 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:34.773279 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcql5" podUID="6cc7e819-7680-4323-8afa-afa00c9433ce" Aug 04 09:40:34.828479 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:34.828436 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdnhg" event={"ID":"4df28d6b-d17f-4a28-b25e-b17559454724","Type":"ContainerStarted","Data":"dcbe5d7bd093d9c6494814b1b07a0da038924d727c7e7c2ab9ca4551bbf13f47"} Aug 04 09:40:35.159943 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:35.159904 2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 09:40:35.793840 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:35.793812 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:40:35.794291 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:35.793968 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjdn8" podUID="b7e0b230-f265-4c9b-9d1d-9008457d8aa1" Aug 04 09:40:36.247418 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:36.247278 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f732a38-dafb-42d2-acc6-ff8059c65350-original-pull-secret\") pod \"global-pull-secret-syncer-7wl64\" (UID: \"8f732a38-dafb-42d2-acc6-ff8059c65350\") " pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:40:36.247583 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:36.247470 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 09:40:36.247583 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:36.247551 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f732a38-dafb-42d2-acc6-ff8059c65350-original-pull-secret podName:8f732a38-dafb-42d2-acc6-ff8059c65350 nodeName:}" failed. No retries permitted until 2026-08-04 09:40:40.247530487 +0000 UTC m=+8.991664904 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f732a38-dafb-42d2-acc6-ff8059c65350-original-pull-secret") pod "global-pull-secret-syncer-7wl64" (UID: "8f732a38-dafb-42d2-acc6-ff8059c65350") : object "kube-system"/"original-pull-secret" not registered Aug 04 09:40:36.349118 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:36.348542 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7e0b230-f265-4c9b-9d1d-9008457d8aa1-metrics-certs\") pod \"network-metrics-daemon-pjdn8\" (UID: \"b7e0b230-f265-4c9b-9d1d-9008457d8aa1\") " pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:40:36.349118 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:36.348698 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:40:36.349118 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:36.348763 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7e0b230-f265-4c9b-9d1d-9008457d8aa1-metrics-certs podName:b7e0b230-f265-4c9b-9d1d-9008457d8aa1 nodeName:}" failed. No retries permitted until 2026-08-04 09:40:40.348743665 +0000 UTC m=+9.092878076 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7e0b230-f265-4c9b-9d1d-9008457d8aa1-metrics-certs") pod "network-metrics-daemon-pjdn8" (UID: "b7e0b230-f265-4c9b-9d1d-9008457d8aa1") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:40:36.449959 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:36.449919 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q78c\" (UniqueName: \"kubernetes.io/projected/6cc7e819-7680-4323-8afa-afa00c9433ce-kube-api-access-4q78c\") pod \"network-check-target-qcql5\" (UID: \"6cc7e819-7680-4323-8afa-afa00c9433ce\") " pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:40:36.450155 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:36.450106 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 09:40:36.450155 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:36.450130 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 09:40:36.450155 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:36.450143 2572 projected.go:196] Error preparing data for projected volume kube-api-access-4q78c for pod openshift-network-diagnostics/network-check-target-qcql5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:40:36.450318 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:36.450200 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6cc7e819-7680-4323-8afa-afa00c9433ce-kube-api-access-4q78c podName:6cc7e819-7680-4323-8afa-afa00c9433ce nodeName:}" failed. No retries permitted until 2026-08-04 09:40:40.450182379 +0000 UTC m=+9.194316780 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4q78c" (UniqueName: "kubernetes.io/projected/6cc7e819-7680-4323-8afa-afa00c9433ce-kube-api-access-4q78c") pod "network-check-target-qcql5" (UID: "6cc7e819-7680-4323-8afa-afa00c9433ce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:40:36.772102 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:36.772062 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:40:36.772297 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:36.772216 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wl64" podUID="8f732a38-dafb-42d2-acc6-ff8059c65350" Aug 04 09:40:36.772662 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:36.772601 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:40:36.772746 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:36.772699 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcql5" podUID="6cc7e819-7680-4323-8afa-afa00c9433ce" Aug 04 09:40:37.772777 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:37.772737 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:40:37.773261 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:37.772936 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjdn8" podUID="b7e0b230-f265-4c9b-9d1d-9008457d8aa1" Aug 04 09:40:38.772783 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:38.772741 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:40:38.773285 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:38.772898 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcql5" podUID="6cc7e819-7680-4323-8afa-afa00c9433ce" Aug 04 09:40:38.773285 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:38.773179 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:40:38.773285 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:38.773274 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wl64" podUID="8f732a38-dafb-42d2-acc6-ff8059c65350" Aug 04 09:40:39.772552 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:39.772517 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:40:39.772737 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:39.772670 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjdn8" podUID="b7e0b230-f265-4c9b-9d1d-9008457d8aa1" Aug 04 09:40:40.283637 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:40.283598 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f732a38-dafb-42d2-acc6-ff8059c65350-original-pull-secret\") pod \"global-pull-secret-syncer-7wl64\" (UID: \"8f732a38-dafb-42d2-acc6-ff8059c65350\") " pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:40:40.284146 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:40.283740 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 09:40:40.284146 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:40.283827 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f732a38-dafb-42d2-acc6-ff8059c65350-original-pull-secret podName:8f732a38-dafb-42d2-acc6-ff8059c65350 nodeName:}" failed. No retries permitted until 2026-08-04 09:40:48.283792359 +0000 UTC m=+17.027926770 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f732a38-dafb-42d2-acc6-ff8059c65350-original-pull-secret") pod "global-pull-secret-syncer-7wl64" (UID: "8f732a38-dafb-42d2-acc6-ff8059c65350") : object "kube-system"/"original-pull-secret" not registered Aug 04 09:40:40.384370 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:40.384337 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7e0b230-f265-4c9b-9d1d-9008457d8aa1-metrics-certs\") pod \"network-metrics-daemon-pjdn8\" (UID: \"b7e0b230-f265-4c9b-9d1d-9008457d8aa1\") " pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:40:40.384553 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:40.384456 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:40:40.384553 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:40.384510 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7e0b230-f265-4c9b-9d1d-9008457d8aa1-metrics-certs podName:b7e0b230-f265-4c9b-9d1d-9008457d8aa1 nodeName:}" failed. No retries permitted until 2026-08-04 09:40:48.384495282 +0000 UTC m=+17.128629680 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7e0b230-f265-4c9b-9d1d-9008457d8aa1-metrics-certs") pod "network-metrics-daemon-pjdn8" (UID: "b7e0b230-f265-4c9b-9d1d-9008457d8aa1") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:40:40.484729 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:40.484660 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q78c\" (UniqueName: \"kubernetes.io/projected/6cc7e819-7680-4323-8afa-afa00c9433ce-kube-api-access-4q78c\") pod \"network-check-target-qcql5\" (UID: \"6cc7e819-7680-4323-8afa-afa00c9433ce\") " pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:40:40.484894 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:40.484810 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 09:40:40.484894 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:40.484836 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 09:40:40.484894 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:40.484849 2572 projected.go:196] Error preparing data for projected volume kube-api-access-4q78c for pod openshift-network-diagnostics/network-check-target-qcql5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:40:40.485007 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:40.484909 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6cc7e819-7680-4323-8afa-afa00c9433ce-kube-api-access-4q78c podName:6cc7e819-7680-4323-8afa-afa00c9433ce nodeName:}" failed. No retries permitted until 2026-08-04 09:40:48.484889537 +0000 UTC m=+17.229023954 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4q78c" (UniqueName: "kubernetes.io/projected/6cc7e819-7680-4323-8afa-afa00c9433ce-kube-api-access-4q78c") pod "network-check-target-qcql5" (UID: "6cc7e819-7680-4323-8afa-afa00c9433ce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:40:40.773004 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:40.772968 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:40:40.773004 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:40.773005 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:40:40.773243 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:40.773113 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wl64" podUID="8f732a38-dafb-42d2-acc6-ff8059c65350" Aug 04 09:40:40.773243 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:40.773199 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcql5" podUID="6cc7e819-7680-4323-8afa-afa00c9433ce" Aug 04 09:40:41.773624 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:41.773128 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:40:41.773624 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:41.773269 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjdn8" podUID="b7e0b230-f265-4c9b-9d1d-9008457d8aa1" Aug 04 09:40:42.772625 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:42.772092 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:40:42.772625 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:42.772240 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wl64" podUID="8f732a38-dafb-42d2-acc6-ff8059c65350" Aug 04 09:40:42.772625 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:42.772300 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:40:42.772625 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:42.772408 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcql5" podUID="6cc7e819-7680-4323-8afa-afa00c9433ce" Aug 04 09:40:43.772156 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:43.772122 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:40:43.772608 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:43.772266 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjdn8" podUID="b7e0b230-f265-4c9b-9d1d-9008457d8aa1" Aug 04 09:40:44.772353 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:44.772316 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:40:44.772767 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:44.772329 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:40:44.772767 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:44.772420 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcql5" podUID="6cc7e819-7680-4323-8afa-afa00c9433ce" Aug 04 09:40:44.772767 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:44.772524 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wl64" podUID="8f732a38-dafb-42d2-acc6-ff8059c65350" Aug 04 09:40:45.772875 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:45.772837 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:40:45.773353 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:45.772974 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjdn8" podUID="b7e0b230-f265-4c9b-9d1d-9008457d8aa1" Aug 04 09:40:46.772063 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:46.772030 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:40:46.772250 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:46.772070 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:40:46.772250 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:46.772146 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wl64" podUID="8f732a38-dafb-42d2-acc6-ff8059c65350" Aug 04 09:40:46.772347 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:46.772264 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcql5" podUID="6cc7e819-7680-4323-8afa-afa00c9433ce" Aug 04 09:40:47.772720 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:47.772688 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:40:47.773208 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:47.772822 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjdn8" podUID="b7e0b230-f265-4c9b-9d1d-9008457d8aa1" Aug 04 09:40:48.342260 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:48.342220 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f732a38-dafb-42d2-acc6-ff8059c65350-original-pull-secret\") pod \"global-pull-secret-syncer-7wl64\" (UID: \"8f732a38-dafb-42d2-acc6-ff8059c65350\") " pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:40:48.342463 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:48.342353 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 09:40:48.342463 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:48.342424 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f732a38-dafb-42d2-acc6-ff8059c65350-original-pull-secret podName:8f732a38-dafb-42d2-acc6-ff8059c65350 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:04.342405317 +0000 UTC m=+33.086539714 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f732a38-dafb-42d2-acc6-ff8059c65350-original-pull-secret") pod "global-pull-secret-syncer-7wl64" (UID: "8f732a38-dafb-42d2-acc6-ff8059c65350") : object "kube-system"/"original-pull-secret" not registered Aug 04 09:40:48.443569 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:48.443526 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7e0b230-f265-4c9b-9d1d-9008457d8aa1-metrics-certs\") pod \"network-metrics-daemon-pjdn8\" (UID: \"b7e0b230-f265-4c9b-9d1d-9008457d8aa1\") " pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:40:48.443757 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:48.443663 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:40:48.443757 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:48.443730 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7e0b230-f265-4c9b-9d1d-9008457d8aa1-metrics-certs podName:b7e0b230-f265-4c9b-9d1d-9008457d8aa1 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:04.443712375 +0000 UTC m=+33.187846792 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7e0b230-f265-4c9b-9d1d-9008457d8aa1-metrics-certs") pod "network-metrics-daemon-pjdn8" (UID: "b7e0b230-f265-4c9b-9d1d-9008457d8aa1") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:40:48.544160 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:48.544111 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q78c\" (UniqueName: \"kubernetes.io/projected/6cc7e819-7680-4323-8afa-afa00c9433ce-kube-api-access-4q78c\") pod \"network-check-target-qcql5\" (UID: \"6cc7e819-7680-4323-8afa-afa00c9433ce\") " pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:40:48.544320 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:48.544274 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 09:40:48.544320 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:48.544298 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 09:40:48.544320 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:48.544308 2572 projected.go:196] Error preparing data for projected volume kube-api-access-4q78c for pod openshift-network-diagnostics/network-check-target-qcql5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:40:48.544440 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:48.544358 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6cc7e819-7680-4323-8afa-afa00c9433ce-kube-api-access-4q78c podName:6cc7e819-7680-4323-8afa-afa00c9433ce nodeName:}" failed. No retries permitted until 2026-08-04 09:41:04.544344786 +0000 UTC m=+33.288479183 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4q78c" (UniqueName: "kubernetes.io/projected/6cc7e819-7680-4323-8afa-afa00c9433ce-kube-api-access-4q78c") pod "network-check-target-qcql5" (UID: "6cc7e819-7680-4323-8afa-afa00c9433ce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:40:48.773085 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:48.773047 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:40:48.773085 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:48.773079 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:40:48.773568 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:48.773190 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wl64" podUID="8f732a38-dafb-42d2-acc6-ff8059c65350" Aug 04 09:40:48.773568 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:48.773324 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcql5" podUID="6cc7e819-7680-4323-8afa-afa00c9433ce" Aug 04 09:40:49.772392 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:49.772195 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:40:49.772579 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:49.772500 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjdn8" podUID="b7e0b230-f265-4c9b-9d1d-9008457d8aa1" Aug 04 09:40:50.773026 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:50.772988 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:40:50.773026 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:50.773011 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:40:50.773518 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:50.773130 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wl64" podUID="8f732a38-dafb-42d2-acc6-ff8059c65350" Aug 04 09:40:50.773518 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:50.773226 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcql5" podUID="6cc7e819-7680-4323-8afa-afa00c9433ce" Aug 04 09:40:51.773249 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:51.773212 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:40:51.773664 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:51.773334 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjdn8" podUID="b7e0b230-f265-4c9b-9d1d-9008457d8aa1" Aug 04 09:40:52.772638 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:52.772599 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:40:52.772857 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:52.772655 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:40:52.772857 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:52.772739 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcql5" podUID="6cc7e819-7680-4323-8afa-afa00c9433ce" Aug 04 09:40:52.773001 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:52.772889 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wl64" podUID="8f732a38-dafb-42d2-acc6-ff8059c65350" Aug 04 09:40:53.772875 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:53.772665 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:40:53.773415 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:53.773010 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjdn8" podUID="b7e0b230-f265-4c9b-9d1d-9008457d8aa1" Aug 04 09:40:53.889102 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:53.889065 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-241.ec2.internal" event={"ID":"88730f4d5c4cac33f5b85009ff17427c","Type":"ContainerStarted","Data":"8936b23d6f074c99e72d193a14c5a445d8d1a00816e040650ff6bc8f576f204b"} Aug 04 09:40:53.890826 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:53.890774 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" event={"ID":"d675e416-204d-4e33-958e-ebdc867294d4","Type":"ContainerStarted","Data":"f926ac60d9ad503bdedfb16c32655545ae2eaf6b0a73f1eddc2c4d1ae299e250"} Aug 04 09:40:53.892334 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:53.892306 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-94bwj" event={"ID":"aa641b90-0c09-4fe6-8fa9-6a0a843144db","Type":"ContainerStarted","Data":"fd9c22867143482c6319644b037eecc5557f5f579ce92ca2bdf27e1d3320c29b"} Aug 04 09:40:53.905623 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:53.905597 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sdsr_7858cfca-73db-4ee1-9882-c13444f112a5/ovn-acl-logging/0.log" Aug 04 09:40:53.906098 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:53.906058 2572 generic.go:356] "Generic (PLEG): container finished" podID="7858cfca-73db-4ee1-9882-c13444f112a5" containerID="f7889f66a0e8f84da9ef64da5630e7b2ee65916d1a31c77928ffcdb0f554f51c" exitCode=1 Aug 04 09:40:53.906202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:53.906119 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" event={"ID":"7858cfca-73db-4ee1-9882-c13444f112a5","Type":"ContainerStarted","Data":"44d512288f610718caf80031d911102f467af9775760deb4d12adb2d473b642d"} Aug 04 09:40:53.906249 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:53.906203 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" event={"ID":"7858cfca-73db-4ee1-9882-c13444f112a5","Type":"ContainerStarted","Data":"0ae614d137b997884b443c650300c93cfb73a56f7090e73b916a90b438d39b2b"} Aug 04 09:40:53.906283 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:53.906249 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" event={"ID":"7858cfca-73db-4ee1-9882-c13444f112a5","Type":"ContainerDied","Data":"f7889f66a0e8f84da9ef64da5630e7b2ee65916d1a31c77928ffcdb0f554f51c"} Aug 04 09:40:53.906283 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:53.906280 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" event={"ID":"7858cfca-73db-4ee1-9882-c13444f112a5","Type":"ContainerStarted","Data":"b4827807ea729f0b72491534612c38f6e6e876eabf9864cb71dce300d14a8c3e"} Aug 04 09:40:53.907008 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:53.906925 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-241.ec2.internal" podStartSLOduration=21.906905357 podStartE2EDuration="21.906905357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:40:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:40:53.906416537 +0000 UTC m=+22.650550955" watchObservedRunningTime="2026-08-04 09:40:53.906905357 +0000 UTC m=+22.651039778" Aug 04 09:40:53.926918 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:53.926873 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6n7bh" podStartSLOduration=2.983239455 podStartE2EDuration="22.926858221s" totalImagesPullingTime="19.943618766s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:40:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:40:53.926224751 +0000 UTC m=+22.670359165" watchObservedRunningTime="2026-08-04 09:40:53.926858221 +0000 UTC m=+22.670992637" Aug 04 09:40:54.772062 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:54.772025 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:40:54.772287 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:54.772029 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:40:54.772287 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:54.772155 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcql5" podUID="6cc7e819-7680-4323-8afa-afa00c9433ce" Aug 04 09:40:54.772287 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:54.772210 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wl64" podUID="8f732a38-dafb-42d2-acc6-ff8059c65350" Aug 04 09:40:54.911018 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:54.910944 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sdsr_7858cfca-73db-4ee1-9882-c13444f112a5/ovn-acl-logging/0.log" Aug 04 09:40:54.911729 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:54.911351 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" event={"ID":"7858cfca-73db-4ee1-9882-c13444f112a5","Type":"ContainerStarted","Data":"1a353f602a81129efac4a946e4524a327d354528859e45cec4bbc4d13c7ac240"} Aug 04 09:40:54.911729 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:54.911438 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" event={"ID":"7858cfca-73db-4ee1-9882-c13444f112a5","Type":"ContainerStarted","Data":"cff03265241abca2760b4c300ab3b88a90a29af5c7190662292c992dc8c9b575"} Aug 04 09:40:54.913403 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:54.913373 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qvwd8" event={"ID":"fa0ec5ee-8ecd-4f22-ae99-cfbe2855395c","Type":"ContainerStarted","Data":"7b8a577e56c03d9806aa0dedf0e00915914ac8e784bee0bb8faaf32958d9ccdb"} Aug 04 09:40:54.915188 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:54.915161 2572 generic.go:356] "Generic (PLEG): container finished" podID="ec918bd2-ac7e-493e-b164-a16ca18c9422" containerID="2664b2b3d50b46de1ca6fabe74750360469b1cf8274fe1451602923207b02ed3" exitCode=0 Aug 04 09:40:54.915315 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:54.915243 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2jnfw" event={"ID":"ec918bd2-ac7e-493e-b164-a16ca18c9422","Type":"ContainerDied","Data":"2664b2b3d50b46de1ca6fabe74750360469b1cf8274fe1451602923207b02ed3"} Aug 04 09:40:54.916783 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:54.916757 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hgcgb" event={"ID":"781f670f-6934-4504-810d-95429f368c7f","Type":"ContainerStarted","Data":"33bd59b6a4cdce3424e08f580991c1c674c5f7c4b36d6c71b64f4990351047c4"} Aug 04 09:40:54.918406 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:54.918379 2572 generic.go:356] "Generic (PLEG): container finished" podID="f0c99023fafad27b36808bb52bb184dd" containerID="a5be5e196af11a7c1aabd8734ba8e5f2b8384ee2aedae2258bb2ef70c4925172" exitCode=0 Aug 04 09:40:54.918502 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:54.918411 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-241.ec2.internal" event={"ID":"f0c99023fafad27b36808bb52bb184dd","Type":"ContainerDied","Data":"a5be5e196af11a7c1aabd8734ba8e5f2b8384ee2aedae2258bb2ef70c4925172"} Aug 04 09:40:54.920144 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:54.920122 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdnhg" event={"ID":"4df28d6b-d17f-4a28-b25e-b17559454724","Type":"ContainerStarted","Data":"fe4f3f0636622240299ec585721c0e657aef2205d5c3f3723226544f570c8e1c"} Aug 04 09:40:54.921706 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:54.921678 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-57rd7" event={"ID":"a9ba7a09-0ff2-4657-8bbd-47b0af96a45a","Type":"ContainerStarted","Data":"551b412513c06245a5796e4b397a474821bec11a3c9ef78a8db01df3efd73749"} Aug 04 09:40:54.923164 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:54.923141 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mbstc" event={"ID":"30012dc7-ffb2-44a7-b0fb-b444100f6e31","Type":"ContainerStarted","Data":"3e712324b15c03c2d272d7499f9f2dcc2b3280c9b5b835fb22dbedbd93bec404"} Aug 04 09:40:54.934105 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:54.934052 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-qvwd8" podStartSLOduration=4.095051925 podStartE2EDuration="23.934033367s" totalImagesPullingTime="19.838981442s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:40:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:40:54.933522376 +0000 UTC m=+23.677656796" watchObservedRunningTime="2026-08-04 09:40:54.934033367 +0000 UTC m=+23.678167788" Aug 04 09:40:54.934245 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:54.934163 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-94bwj" podStartSLOduration=3.056063054 podStartE2EDuration="22.934157363s" totalImagesPullingTime="19.878094309s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:40:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:40:53.950234118 +0000 UTC m=+22.694368536" watchObservedRunningTime="2026-08-04 09:40:54.934157363 +0000 UTC m=+23.678291784" Aug 04 09:40:54.999513 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:54.999339 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mbstc" podStartSLOduration=3.075984784 podStartE2EDuration="22.999323855s" totalImagesPullingTime="19.923339071s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:40:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:40:54.978678103 +0000 UTC m=+23.722812526" watchObservedRunningTime="2026-08-04 09:40:54.999323855 +0000 UTC m=+23.743458274" Aug 04 09:40:55.037404 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:55.037330 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-57rd7" podStartSLOduration=4.279238387 podStartE2EDuration="24.037314109s" totalImagesPullingTime="19.758075722s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:40:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:40:55.036996882 +0000 UTC m=+23.781131300" watchObservedRunningTime="2026-08-04 09:40:55.037314109 +0000 UTC m=+23.781448533" Aug 04 09:40:55.037575 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:55.037533 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hgcgb" podStartSLOduration=8.257854243 podStartE2EDuration="24.037526843s" totalImagesPullingTime="15.7796726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:40:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:40:55.02217684 +0000 UTC m=+23.766311258" watchObservedRunningTime="2026-08-04 09:40:55.037526843 +0000 UTC m=+23.781661336" Aug 04 09:40:55.594495 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:55.594465 2572 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Aug 04 09:40:55.681412 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:55.681317 2572 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-04T09:40:55.594488061Z","UUID":"14f8de31-740c-4bbf-aab4-865978481182","Handler":null,"Name":"","Endpoint":""} Aug 04 09:40:55.682876 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:55.682856 2572 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Aug 04 09:40:55.682990 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:55.682890 2572 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Aug 04 09:40:55.772214 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:55.772129 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:40:55.772393 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:55.772271 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjdn8" podUID="b7e0b230-f265-4c9b-9d1d-9008457d8aa1" Aug 04 09:40:55.882515 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:55.882484 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:40:55.926865 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:55.926830 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-241.ec2.internal" event={"ID":"f0c99023fafad27b36808bb52bb184dd","Type":"ContainerStarted","Data":"2086831a0d76844c02025512e9e00c09029b5f4aeebb72b8ac4d7f2a28ea66f9"} Aug 04 09:40:55.928664 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:55.928643 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdnhg" event={"ID":"4df28d6b-d17f-4a28-b25e-b17559454724","Type":"ContainerStarted","Data":"f74ea191dc0200712f4a0838c589a0b297a8ddf009085545aa18cb6d99281f6d"} Aug 04 09:40:55.945744 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:55.945691 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-241.ec2.internal" podStartSLOduration=23.945674864 podStartE2EDuration="23.945674864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:40:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:40:55.945199817 +0000 UTC m=+24.689334235" watchObservedRunningTime="2026-08-04 09:40:55.945674864 +0000 UTC m=+24.689809283" Aug 04 09:40:56.004078 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:56.004044 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:40:56.243517 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:56.243481 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:40:56.340837 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:56.340735 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:40:56.773110 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:56.773069 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:40:56.773428 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:56.773083 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:40:56.773428 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:56.773181 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcql5" podUID="6cc7e819-7680-4323-8afa-afa00c9433ce" Aug 04 09:40:56.773428 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:56.773245 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wl64" podUID="8f732a38-dafb-42d2-acc6-ff8059c65350" Aug 04 09:40:56.935377 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:56.935354 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sdsr_7858cfca-73db-4ee1-9882-c13444f112a5/ovn-acl-logging/0.log" Aug 04 09:40:56.935826 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:56.935782 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" event={"ID":"7858cfca-73db-4ee1-9882-c13444f112a5","Type":"ContainerStarted","Data":"30a481f3d63463e6f2b2ccbc9289a9cb60e033cfda00712edaaf5728179bb44d"} Aug 04 09:40:57.351336 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:57.351303 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hgcgb" Aug 04 09:40:57.352104 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:57.352075 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hgcgb" Aug 04 09:40:57.775724 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:57.775680 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:40:57.775907 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:57.775867 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjdn8" podUID="b7e0b230-f265-4c9b-9d1d-9008457d8aa1" Aug 04 09:40:57.940290 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:57.940257 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdnhg" event={"ID":"4df28d6b-d17f-4a28-b25e-b17559454724","Type":"ContainerStarted","Data":"8001871bea2a4a79a4bdaa609cdac905f68dd8dc76f2dc2e0516ec5023d3ebe4"} Aug 04 09:40:57.961039 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:57.960988 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mdnhg" podStartSLOduration=4.319079487 podStartE2EDuration="25.960973351s" totalImagesPullingTime="21.641893864s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:40:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:40:57.960474095 +0000 UTC m=+26.704608514" watchObservedRunningTime="2026-08-04 09:40:57.960973351 +0000 UTC m=+26.705107769" Aug 04 09:40:58.773048 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:58.773016 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:40:58.773230 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:58.773018 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:40:58.773230 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:58.773130 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wl64" podUID="8f732a38-dafb-42d2-acc6-ff8059c65350" Aug 04 09:40:58.773230 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:58.773208 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcql5" podUID="6cc7e819-7680-4323-8afa-afa00c9433ce" Aug 04 09:40:58.908684 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:58.908650 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:40:59.005024 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:59.004989 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:40:59.449232 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:59.449195 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:40:59.576635 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:59.576471 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:40:59.776305 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:59.776282 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:40:59.776532 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:40:59.776392 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjdn8" podUID="b7e0b230-f265-4c9b-9d1d-9008457d8aa1" Aug 04 09:40:59.948312 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:59.948288 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sdsr_7858cfca-73db-4ee1-9882-c13444f112a5/ovn-acl-logging/0.log" Aug 04 09:40:59.948657 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:59.948619 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" event={"ID":"7858cfca-73db-4ee1-9882-c13444f112a5","Type":"ContainerStarted","Data":"9f6b4563469cd228c476b4a94545d197316a4da08efee2b41885eaed2a204195"} Aug 04 09:40:59.949038 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:59.949018 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:59.949127 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:59.949104 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:40:59.949175 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:59.949158 2572 scope.go:117] "RemoveContainer" containerID="f7889f66a0e8f84da9ef64da5630e7b2ee65916d1a31c77928ffcdb0f554f51c" Aug 04 09:40:59.950602 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:59.950578 2572 generic.go:356] "Generic (PLEG): container finished" podID="ec918bd2-ac7e-493e-b164-a16ca18c9422" containerID="0d52476a16fec49d94b744665934563dfe4e71289ba5699976570d3ac25252fa" exitCode=0 Aug 04 09:40:59.950672 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:59.950620 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2jnfw" event={"ID":"ec918bd2-ac7e-493e-b164-a16ca18c9422","Type":"ContainerDied","Data":"0d52476a16fec49d94b744665934563dfe4e71289ba5699976570d3ac25252fa"} Aug 04 09:40:59.965311 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:40:59.965287 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:41:00.772373 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:00.772340 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:41:00.772373 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:00.772360 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:41:00.773375 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:00.772465 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wl64" podUID="8f732a38-dafb-42d2-acc6-ff8059c65350" Aug 04 09:41:00.773375 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:00.772606 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcql5" podUID="6cc7e819-7680-4323-8afa-afa00c9433ce" Aug 04 09:41:00.957342 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:00.957311 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sdsr_7858cfca-73db-4ee1-9882-c13444f112a5/ovn-acl-logging/0.log" Aug 04 09:41:00.957669 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:00.957646 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" event={"ID":"7858cfca-73db-4ee1-9882-c13444f112a5","Type":"ContainerStarted","Data":"989d36362c593212086ff8d4ad3744e8bf20bca0e70b01cb570a472611cad4f2"} Aug 04 09:41:00.957938 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:00.957915 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:41:00.973509 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:00.973481 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:41:00.993503 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:00.993451 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" podStartSLOduration=10.159183141 podStartE2EDuration="29.993434837s" totalImagesPullingTime="19.834251696s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:40:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:00.99312444 +0000 UTC m=+29.737258871" watchObservedRunningTime="2026-08-04 09:41:00.993434837 +0000 UTC m=+29.737569255" Aug 04 09:41:01.663785 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:01.663552 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7wl64"] Aug 04 09:41:01.663958 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:01.663907 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:41:01.664060 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:01.664035 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wl64" podUID="8f732a38-dafb-42d2-acc6-ff8059c65350" Aug 04 09:41:01.667327 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:01.667292 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pjdn8"] Aug 04 09:41:01.667483 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:01.667435 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:41:01.667604 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:01.667553 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjdn8" podUID="b7e0b230-f265-4c9b-9d1d-9008457d8aa1" Aug 04 09:41:01.668008 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:01.667988 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qcql5"] Aug 04 09:41:01.668099 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:01.668087 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:41:01.668176 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:01.668159 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcql5" podUID="6cc7e819-7680-4323-8afa-afa00c9433ce" Aug 04 09:41:01.962274 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:01.962242 2572 generic.go:356] "Generic (PLEG): container finished" podID="ec918bd2-ac7e-493e-b164-a16ca18c9422" containerID="267420e32b22cfa40ae7e54cacd5d2732ff06d7b6c7a83d41c534608b00d418e" exitCode=0 Aug 04 09:41:01.962673 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:01.962333 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2jnfw" event={"ID":"ec918bd2-ac7e-493e-b164-a16ca18c9422","Type":"ContainerDied","Data":"267420e32b22cfa40ae7e54cacd5d2732ff06d7b6c7a83d41c534608b00d418e"} Aug 04 09:41:02.591476 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:02.591386 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hgcgb" Aug 04 09:41:02.591613 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:02.591516 2572 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Aug 04 09:41:02.592050 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:02.592029 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hgcgb" Aug 04 09:41:02.663101 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:02.663060 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:03.424902 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:03.424822 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:03.772581 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:03.772545 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:41:03.772762 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:03.772556 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:41:03.772762 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:03.772645 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wl64" podUID="8f732a38-dafb-42d2-acc6-ff8059c65350" Aug 04 09:41:03.772762 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:03.772671 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:41:03.772937 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:03.772782 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjdn8" podUID="b7e0b230-f265-4c9b-9d1d-9008457d8aa1" Aug 04 09:41:03.772937 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:03.772908 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcql5" podUID="6cc7e819-7680-4323-8afa-afa00c9433ce" Aug 04 09:41:03.971722 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:03.971687 2572 generic.go:356] "Generic (PLEG): container finished" podID="ec918bd2-ac7e-493e-b164-a16ca18c9422" containerID="c7afbee9b750dc0295ae8b86e88e44bf83559ec9fb20f0b11bdc79946050f8e1" exitCode=0 Aug 04 09:41:03.971722 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:03.971724 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2jnfw" event={"ID":"ec918bd2-ac7e-493e-b164-a16ca18c9422","Type":"ContainerDied","Data":"c7afbee9b750dc0295ae8b86e88e44bf83559ec9fb20f0b11bdc79946050f8e1"} Aug 04 09:41:04.359557 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:04.359455 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f732a38-dafb-42d2-acc6-ff8059c65350-original-pull-secret\") pod \"global-pull-secret-syncer-7wl64\" (UID: \"8f732a38-dafb-42d2-acc6-ff8059c65350\") " pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:41:04.359557 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:04.359551 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 09:41:04.359807 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:04.359626 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f732a38-dafb-42d2-acc6-ff8059c65350-original-pull-secret podName:8f732a38-dafb-42d2-acc6-ff8059c65350 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:36.359606282 +0000 UTC m=+65.103740694 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f732a38-dafb-42d2-acc6-ff8059c65350-original-pull-secret") pod "global-pull-secret-syncer-7wl64" (UID: "8f732a38-dafb-42d2-acc6-ff8059c65350") : object "kube-system"/"original-pull-secret" not registered Aug 04 09:41:04.460470 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:04.460437 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7e0b230-f265-4c9b-9d1d-9008457d8aa1-metrics-certs\") pod \"network-metrics-daemon-pjdn8\" (UID: \"b7e0b230-f265-4c9b-9d1d-9008457d8aa1\") " pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:41:04.460887 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:04.460592 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:41:04.460887 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:04.460664 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7e0b230-f265-4c9b-9d1d-9008457d8aa1-metrics-certs podName:b7e0b230-f265-4c9b-9d1d-9008457d8aa1 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:36.460639821 +0000 UTC m=+65.204774241 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b7e0b230-f265-4c9b-9d1d-9008457d8aa1-metrics-certs") pod "network-metrics-daemon-pjdn8" (UID: "b7e0b230-f265-4c9b-9d1d-9008457d8aa1") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 09:41:04.561230 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:04.561190 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q78c\" (UniqueName: \"kubernetes.io/projected/6cc7e819-7680-4323-8afa-afa00c9433ce-kube-api-access-4q78c\") pod \"network-check-target-qcql5\" (UID: \"6cc7e819-7680-4323-8afa-afa00c9433ce\") " pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:41:04.561398 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:04.561320 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 09:41:04.561398 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:04.561334 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 09:41:04.561398 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:04.561342 2572 projected.go:196] Error preparing data for projected volume kube-api-access-4q78c for pod openshift-network-diagnostics/network-check-target-qcql5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:41:04.561398 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:04.561389 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6cc7e819-7680-4323-8afa-afa00c9433ce-kube-api-access-4q78c podName:6cc7e819-7680-4323-8afa-afa00c9433ce nodeName:}" failed. No retries permitted until 2026-08-04 09:41:36.561374908 +0000 UTC m=+65.305509321 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4q78c" (UniqueName: "kubernetes.io/projected/6cc7e819-7680-4323-8afa-afa00c9433ce-kube-api-access-4q78c") pod "network-check-target-qcql5" (UID: "6cc7e819-7680-4323-8afa-afa00c9433ce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 09:41:05.773790 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:05.773754 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:41:05.773790 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:05.773791 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:41:05.773790 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:05.773752 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:41:05.774516 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:05.774316 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qcql5" podUID="6cc7e819-7680-4323-8afa-afa00c9433ce" Aug 04 09:41:05.774516 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:05.774363 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7wl64" podUID="8f732a38-dafb-42d2-acc6-ff8059c65350" Aug 04 09:41:05.774516 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:05.774462 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pjdn8" podUID="b7e0b230-f265-4c9b-9d1d-9008457d8aa1" Aug 04 09:41:06.085505 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.085326 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-241.ec2.internal" event="NodeReady" Aug 04 09:41:06.085664 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.085609 2572 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Aug 04 09:41:06.139885 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.139848 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cmck8"] Aug 04 09:41:06.141904 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.141870 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cmck8" Aug 04 09:41:06.145183 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.145157 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Aug 04 09:41:06.145320 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.145307 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Aug 04 09:41:06.145425 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.145394 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-psh6m\"" Aug 04 09:41:06.145557 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.145532 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-z68g5"] Aug 04 09:41:06.147458 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.147436 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z68g5" Aug 04 09:41:06.150435 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.150416 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Aug 04 09:41:06.150563 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.150418 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rgp27\"" Aug 04 09:41:06.150721 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.150704 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Aug 04 09:41:06.150978 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.150955 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Aug 04 09:41:06.160351 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.160279 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cmck8"] Aug 04 09:41:06.162142 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.162122 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z68g5"] Aug 04 09:41:06.275511 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.275477 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d4ecc98-460d-41bb-9fb6-fcea7a43d789-cert\") pod \"ingress-canary-z68g5\" (UID: \"3d4ecc98-460d-41bb-9fb6-fcea7a43d789\") " pod="openshift-ingress-canary/ingress-canary-z68g5" Aug 04 09:41:06.275671 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.275521 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nhc6\" (UniqueName: \"kubernetes.io/projected/5cce2515-bbb6-4fc5-b6ee-9825796843f2-kube-api-access-9nhc6\") pod \"dns-default-cmck8\" (UID: \"5cce2515-bbb6-4fc5-b6ee-9825796843f2\") " pod="openshift-dns/dns-default-cmck8" Aug 04 09:41:06.275671 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.275600 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5cce2515-bbb6-4fc5-b6ee-9825796843f2-metrics-tls\") pod \"dns-default-cmck8\" (UID: \"5cce2515-bbb6-4fc5-b6ee-9825796843f2\") " pod="openshift-dns/dns-default-cmck8" Aug 04 09:41:06.275671 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.275635 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmh49\" (UniqueName: \"kubernetes.io/projected/3d4ecc98-460d-41bb-9fb6-fcea7a43d789-kube-api-access-qmh49\") pod \"ingress-canary-z68g5\" (UID: \"3d4ecc98-460d-41bb-9fb6-fcea7a43d789\") " pod="openshift-ingress-canary/ingress-canary-z68g5" Aug 04 09:41:06.275866 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.275685 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cce2515-bbb6-4fc5-b6ee-9825796843f2-tmp-dir\") pod \"dns-default-cmck8\" (UID: \"5cce2515-bbb6-4fc5-b6ee-9825796843f2\") " pod="openshift-dns/dns-default-cmck8" Aug 04 09:41:06.275866 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.275755 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5cce2515-bbb6-4fc5-b6ee-9825796843f2-config-volume\") pod \"dns-default-cmck8\" (UID: \"5cce2515-bbb6-4fc5-b6ee-9825796843f2\") " pod="openshift-dns/dns-default-cmck8" Aug 04 09:41:06.376278 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.376185 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d4ecc98-460d-41bb-9fb6-fcea7a43d789-cert\") pod \"ingress-canary-z68g5\" (UID: \"3d4ecc98-460d-41bb-9fb6-fcea7a43d789\") " pod="openshift-ingress-canary/ingress-canary-z68g5" Aug 04 09:41:06.376278 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.376229 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nhc6\" (UniqueName: \"kubernetes.io/projected/5cce2515-bbb6-4fc5-b6ee-9825796843f2-kube-api-access-9nhc6\") pod \"dns-default-cmck8\" (UID: \"5cce2515-bbb6-4fc5-b6ee-9825796843f2\") " pod="openshift-dns/dns-default-cmck8" Aug 04 09:41:06.376278 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.376264 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5cce2515-bbb6-4fc5-b6ee-9825796843f2-metrics-tls\") pod \"dns-default-cmck8\" (UID: \"5cce2515-bbb6-4fc5-b6ee-9825796843f2\") " pod="openshift-dns/dns-default-cmck8" Aug 04 09:41:06.376514 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.376291 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmh49\" (UniqueName: \"kubernetes.io/projected/3d4ecc98-460d-41bb-9fb6-fcea7a43d789-kube-api-access-qmh49\") pod \"ingress-canary-z68g5\" (UID: \"3d4ecc98-460d-41bb-9fb6-fcea7a43d789\") " pod="openshift-ingress-canary/ingress-canary-z68g5" Aug 04 09:41:06.376514 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.376325 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cce2515-bbb6-4fc5-b6ee-9825796843f2-tmp-dir\") pod \"dns-default-cmck8\" (UID: \"5cce2515-bbb6-4fc5-b6ee-9825796843f2\") " pod="openshift-dns/dns-default-cmck8" Aug 04 09:41:06.376514 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.376354 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5cce2515-bbb6-4fc5-b6ee-9825796843f2-config-volume\") pod \"dns-default-cmck8\" (UID: \"5cce2515-bbb6-4fc5-b6ee-9825796843f2\") " pod="openshift-dns/dns-default-cmck8" Aug 04 09:41:06.376766 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:06.376738 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 09:41:06.376895 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:06.376833 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cce2515-bbb6-4fc5-b6ee-9825796843f2-metrics-tls podName:5cce2515-bbb6-4fc5-b6ee-9825796843f2 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:06.876791203 +0000 UTC m=+35.620925614 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5cce2515-bbb6-4fc5-b6ee-9825796843f2-metrics-tls") pod "dns-default-cmck8" (UID: "5cce2515-bbb6-4fc5-b6ee-9825796843f2") : secret "dns-default-metrics-tls" not found Aug 04 09:41:06.376987 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.376926 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cce2515-bbb6-4fc5-b6ee-9825796843f2-tmp-dir\") pod \"dns-default-cmck8\" (UID: \"5cce2515-bbb6-4fc5-b6ee-9825796843f2\") " pod="openshift-dns/dns-default-cmck8" Aug 04 09:41:06.377125 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.377109 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5cce2515-bbb6-4fc5-b6ee-9825796843f2-config-volume\") pod \"dns-default-cmck8\" (UID: \"5cce2515-bbb6-4fc5-b6ee-9825796843f2\") " pod="openshift-dns/dns-default-cmck8" Aug 04 09:41:06.381126 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.381105 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d4ecc98-460d-41bb-9fb6-fcea7a43d789-cert\") pod \"ingress-canary-z68g5\" (UID: \"3d4ecc98-460d-41bb-9fb6-fcea7a43d789\") " pod="openshift-ingress-canary/ingress-canary-z68g5" Aug 04 09:41:06.389148 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.389094 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nhc6\" (UniqueName: \"kubernetes.io/projected/5cce2515-bbb6-4fc5-b6ee-9825796843f2-kube-api-access-9nhc6\") pod \"dns-default-cmck8\" (UID: \"5cce2515-bbb6-4fc5-b6ee-9825796843f2\") " pod="openshift-dns/dns-default-cmck8" Aug 04 09:41:06.389530 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.389503 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmh49\" (UniqueName: \"kubernetes.io/projected/3d4ecc98-460d-41bb-9fb6-fcea7a43d789-kube-api-access-qmh49\") pod \"ingress-canary-z68g5\" (UID: \"3d4ecc98-460d-41bb-9fb6-fcea7a43d789\") " pod="openshift-ingress-canary/ingress-canary-z68g5" Aug 04 09:41:06.461012 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.460978 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z68g5" Aug 04 09:41:06.880220 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:06.880176 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5cce2515-bbb6-4fc5-b6ee-9825796843f2-metrics-tls\") pod \"dns-default-cmck8\" (UID: \"5cce2515-bbb6-4fc5-b6ee-9825796843f2\") " pod="openshift-dns/dns-default-cmck8" Aug 04 09:41:06.880705 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:06.880345 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 09:41:06.880705 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:06.880415 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cce2515-bbb6-4fc5-b6ee-9825796843f2-metrics-tls podName:5cce2515-bbb6-4fc5-b6ee-9825796843f2 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:07.880399254 +0000 UTC m=+36.624533673 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5cce2515-bbb6-4fc5-b6ee-9825796843f2-metrics-tls") pod "dns-default-cmck8" (UID: "5cce2515-bbb6-4fc5-b6ee-9825796843f2") : secret "dns-default-metrics-tls" not found Aug 04 09:41:07.468599 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.468562 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-kptzz"] Aug 04 09:41:07.470712 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.470683 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-kptzz" Aug 04 09:41:07.473039 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.473013 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-69f7b748fb-vt9zm"] Aug 04 09:41:07.473832 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.473725 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Aug 04 09:41:07.474036 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.474016 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Aug 04 09:41:07.474290 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.474272 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-kz4kj\"" Aug 04 09:41:07.474936 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.474917 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:07.477835 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.477813 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Aug 04 09:41:07.477941 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.477841 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Aug 04 09:41:07.477941 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.477870 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Aug 04 09:41:07.478408 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.478128 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Aug 04 09:41:07.478408 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.478148 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-95kdr\"" Aug 04 09:41:07.478408 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.478149 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Aug 04 09:41:07.478408 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.478151 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Aug 04 09:41:07.484658 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.484630 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-kptzz"] Aug 04 09:41:07.489628 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.489598 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-69f7b748fb-vt9zm"] Aug 04 09:41:07.584693 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.584656 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee459213-237c-404d-aa9d-8f681ee6e595-service-ca-bundle\") pod \"router-default-69f7b748fb-vt9zm\" (UID: \"ee459213-237c-404d-aa9d-8f681ee6e595\") " pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:07.584693 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.584700 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ee459213-237c-404d-aa9d-8f681ee6e595-stats-auth\") pod \"router-default-69f7b748fb-vt9zm\" (UID: \"ee459213-237c-404d-aa9d-8f681ee6e595\") " pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:07.584917 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.584818 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee459213-237c-404d-aa9d-8f681ee6e595-metrics-certs\") pod \"router-default-69f7b748fb-vt9zm\" (UID: \"ee459213-237c-404d-aa9d-8f681ee6e595\") " pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:07.584917 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.584851 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvn9t\" (UniqueName: \"kubernetes.io/projected/ee459213-237c-404d-aa9d-8f681ee6e595-kube-api-access-pvn9t\") pod \"router-default-69f7b748fb-vt9zm\" (UID: \"ee459213-237c-404d-aa9d-8f681ee6e595\") " pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:07.584917 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.584878 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js5zd\" (UniqueName: \"kubernetes.io/projected/ef014102-c117-4458-8b5c-169ae01b6d70-kube-api-access-js5zd\") pod \"volume-data-source-validator-5b8776669-kptzz\" (UID: \"ef014102-c117-4458-8b5c-169ae01b6d70\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-kptzz" Aug 04 09:41:07.585046 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.584948 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ee459213-237c-404d-aa9d-8f681ee6e595-default-certificate\") pod \"router-default-69f7b748fb-vt9zm\" (UID: \"ee459213-237c-404d-aa9d-8f681ee6e595\") " pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:07.686169 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.686130 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee459213-237c-404d-aa9d-8f681ee6e595-service-ca-bundle\") pod \"router-default-69f7b748fb-vt9zm\" (UID: \"ee459213-237c-404d-aa9d-8f681ee6e595\") " pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:07.686364 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.686288 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ee459213-237c-404d-aa9d-8f681ee6e595-stats-auth\") pod \"router-default-69f7b748fb-vt9zm\" (UID: \"ee459213-237c-404d-aa9d-8f681ee6e595\") " pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:07.686364 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:07.686300 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ee459213-237c-404d-aa9d-8f681ee6e595-service-ca-bundle podName:ee459213-237c-404d-aa9d-8f681ee6e595 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:08.186275466 +0000 UTC m=+36.930409864 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ee459213-237c-404d-aa9d-8f681ee6e595-service-ca-bundle") pod "router-default-69f7b748fb-vt9zm" (UID: "ee459213-237c-404d-aa9d-8f681ee6e595") : configmap references non-existent config key: service-ca.crt Aug 04 09:41:07.686501 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.686395 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee459213-237c-404d-aa9d-8f681ee6e595-metrics-certs\") pod \"router-default-69f7b748fb-vt9zm\" (UID: \"ee459213-237c-404d-aa9d-8f681ee6e595\") " pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:07.686501 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.686432 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvn9t\" (UniqueName: \"kubernetes.io/projected/ee459213-237c-404d-aa9d-8f681ee6e595-kube-api-access-pvn9t\") pod \"router-default-69f7b748fb-vt9zm\" (UID: \"ee459213-237c-404d-aa9d-8f681ee6e595\") " pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:07.686501 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.686464 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js5zd\" (UniqueName: \"kubernetes.io/projected/ef014102-c117-4458-8b5c-169ae01b6d70-kube-api-access-js5zd\") pod \"volume-data-source-validator-5b8776669-kptzz\" (UID: \"ef014102-c117-4458-8b5c-169ae01b6d70\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-kptzz" Aug 04 09:41:07.686638 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.686556 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ee459213-237c-404d-aa9d-8f681ee6e595-default-certificate\") pod \"router-default-69f7b748fb-vt9zm\" (UID: \"ee459213-237c-404d-aa9d-8f681ee6e595\") " pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:07.686638 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:07.686586 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Aug 04 09:41:07.686738 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:07.686646 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee459213-237c-404d-aa9d-8f681ee6e595-metrics-certs podName:ee459213-237c-404d-aa9d-8f681ee6e595 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:08.186627599 +0000 UTC m=+36.930762011 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ee459213-237c-404d-aa9d-8f681ee6e595-metrics-certs") pod "router-default-69f7b748fb-vt9zm" (UID: "ee459213-237c-404d-aa9d-8f681ee6e595") : secret "router-metrics-certs-default" not found Aug 04 09:41:07.689167 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.689142 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ee459213-237c-404d-aa9d-8f681ee6e595-default-certificate\") pod \"router-default-69f7b748fb-vt9zm\" (UID: \"ee459213-237c-404d-aa9d-8f681ee6e595\") " pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:07.689292 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.689232 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ee459213-237c-404d-aa9d-8f681ee6e595-stats-auth\") pod \"router-default-69f7b748fb-vt9zm\" (UID: \"ee459213-237c-404d-aa9d-8f681ee6e595\") " pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:07.696966 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.696941 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-js5zd\" (UniqueName: \"kubernetes.io/projected/ef014102-c117-4458-8b5c-169ae01b6d70-kube-api-access-js5zd\") pod \"volume-data-source-validator-5b8776669-kptzz\" (UID: \"ef014102-c117-4458-8b5c-169ae01b6d70\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-kptzz" Aug 04 09:41:07.697482 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.697453 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvn9t\" (UniqueName: \"kubernetes.io/projected/ee459213-237c-404d-aa9d-8f681ee6e595-kube-api-access-pvn9t\") pod \"router-default-69f7b748fb-vt9zm\" (UID: \"ee459213-237c-404d-aa9d-8f681ee6e595\") " pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:07.729358 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.729278 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:07.772072 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.772037 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:41:07.772260 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.772037 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:41:07.772260 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.772037 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:41:07.782822 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.782781 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Aug 04 09:41:07.783070 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.783053 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Aug 04 09:41:07.783170 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.783121 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zpz8l\"" Aug 04 09:41:07.783346 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.783313 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Aug 04 09:41:07.783466 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.783395 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gfsnh\"" Aug 04 09:41:07.783640 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.783624 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-kptzz" Aug 04 09:41:07.784189 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.784171 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Aug 04 09:41:07.828466 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.828431 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:07.888037 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.887926 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5cce2515-bbb6-4fc5-b6ee-9825796843f2-metrics-tls\") pod \"dns-default-cmck8\" (UID: \"5cce2515-bbb6-4fc5-b6ee-9825796843f2\") " pod="openshift-dns/dns-default-cmck8" Aug 04 09:41:07.888846 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:07.888409 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 09:41:07.888846 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:07.888480 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cce2515-bbb6-4fc5-b6ee-9825796843f2-metrics-tls podName:5cce2515-bbb6-4fc5-b6ee-9825796843f2 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:09.888458286 +0000 UTC m=+38.632592703 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5cce2515-bbb6-4fc5-b6ee-9825796843f2-metrics-tls") pod "dns-default-cmck8" (UID: "5cce2515-bbb6-4fc5-b6ee-9825796843f2") : secret "dns-default-metrics-tls" not found Aug 04 09:41:07.972144 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.971770 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z68g5"] Aug 04 09:41:07.974175 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.974152 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-kptzz"] Aug 04 09:41:07.975016 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:41:07.974994 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d4ecc98_460d_41bb_9fb6_fcea7a43d789.slice/crio-e34d6cf2c78598604ff16b0116c4d7ea8b48fdeb10ad6effcdb323eb9eb89fda WatchSource:0}: Error finding container e34d6cf2c78598604ff16b0116c4d7ea8b48fdeb10ad6effcdb323eb9eb89fda: Status 404 returned error can't find the container with id e34d6cf2c78598604ff16b0116c4d7ea8b48fdeb10ad6effcdb323eb9eb89fda Aug 04 09:41:07.976232 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:41:07.976212 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef014102_c117_4458_8b5c_169ae01b6d70.slice/crio-9f4c6416fe15dbfd4e374402687df11d7d57019dfeee3fdce9d90f465913dda1 WatchSource:0}: Error finding container 9f4c6416fe15dbfd4e374402687df11d7d57019dfeee3fdce9d90f465913dda1: Status 404 returned error can't find the container with id 9f4c6416fe15dbfd4e374402687df11d7d57019dfeee3fdce9d90f465913dda1 Aug 04 09:41:07.985290 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.985259 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2jnfw" event={"ID":"ec918bd2-ac7e-493e-b164-a16ca18c9422","Type":"ContainerStarted","Data":"a072ef10725db3de337f37974420341f0704dcd3dc3205afa2b0c11c658c6ea9"} Aug 04 09:41:07.986342 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.986321 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-kptzz" event={"ID":"ef014102-c117-4458-8b5c-169ae01b6d70","Type":"ContainerStarted","Data":"9f4c6416fe15dbfd4e374402687df11d7d57019dfeee3fdce9d90f465913dda1"} Aug 04 09:41:07.987498 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:07.987475 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z68g5" event={"ID":"3d4ecc98-460d-41bb-9fb6-fcea7a43d789","Type":"ContainerStarted","Data":"e34d6cf2c78598604ff16b0116c4d7ea8b48fdeb10ad6effcdb323eb9eb89fda"} Aug 04 09:41:08.190819 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:08.190759 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee459213-237c-404d-aa9d-8f681ee6e595-service-ca-bundle\") pod \"router-default-69f7b748fb-vt9zm\" (UID: \"ee459213-237c-404d-aa9d-8f681ee6e595\") " pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:08.190989 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:08.190868 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee459213-237c-404d-aa9d-8f681ee6e595-metrics-certs\") pod \"router-default-69f7b748fb-vt9zm\" (UID: \"ee459213-237c-404d-aa9d-8f681ee6e595\") " pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:08.190989 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:08.190945 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ee459213-237c-404d-aa9d-8f681ee6e595-service-ca-bundle podName:ee459213-237c-404d-aa9d-8f681ee6e595 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:09.190926501 +0000 UTC m=+37.935060919 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ee459213-237c-404d-aa9d-8f681ee6e595-service-ca-bundle") pod "router-default-69f7b748fb-vt9zm" (UID: "ee459213-237c-404d-aa9d-8f681ee6e595") : configmap references non-existent config key: service-ca.crt Aug 04 09:41:08.191070 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:08.191022 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Aug 04 09:41:08.191103 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:08.191074 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee459213-237c-404d-aa9d-8f681ee6e595-metrics-certs podName:ee459213-237c-404d-aa9d-8f681ee6e595 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:09.191061255 +0000 UTC m=+37.935195680 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ee459213-237c-404d-aa9d-8f681ee6e595-metrics-certs") pod "router-default-69f7b748fb-vt9zm" (UID: "ee459213-237c-404d-aa9d-8f681ee6e595") : secret "router-metrics-certs-default" not found Aug 04 09:41:08.994543 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:08.994502 2572 generic.go:356] "Generic (PLEG): container finished" podID="ec918bd2-ac7e-493e-b164-a16ca18c9422" containerID="a072ef10725db3de337f37974420341f0704dcd3dc3205afa2b0c11c658c6ea9" exitCode=0 Aug 04 09:41:08.995024 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:08.994558 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2jnfw" event={"ID":"ec918bd2-ac7e-493e-b164-a16ca18c9422","Type":"ContainerDied","Data":"a072ef10725db3de337f37974420341f0704dcd3dc3205afa2b0c11c658c6ea9"} Aug 04 09:41:09.198244 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:09.198186 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee459213-237c-404d-aa9d-8f681ee6e595-service-ca-bundle\") pod \"router-default-69f7b748fb-vt9zm\" (UID: \"ee459213-237c-404d-aa9d-8f681ee6e595\") " pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:09.198533 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:09.198278 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee459213-237c-404d-aa9d-8f681ee6e595-metrics-certs\") pod \"router-default-69f7b748fb-vt9zm\" (UID: \"ee459213-237c-404d-aa9d-8f681ee6e595\") " pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:09.198533 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:09.198442 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Aug 04 09:41:09.198533 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:09.198504 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee459213-237c-404d-aa9d-8f681ee6e595-metrics-certs podName:ee459213-237c-404d-aa9d-8f681ee6e595 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:11.198484693 +0000 UTC m=+39.942619108 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ee459213-237c-404d-aa9d-8f681ee6e595-metrics-certs") pod "router-default-69f7b748fb-vt9zm" (UID: "ee459213-237c-404d-aa9d-8f681ee6e595") : secret "router-metrics-certs-default" not found Aug 04 09:41:09.199004 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:09.198968 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ee459213-237c-404d-aa9d-8f681ee6e595-service-ca-bundle podName:ee459213-237c-404d-aa9d-8f681ee6e595 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:11.198948807 +0000 UTC m=+39.943083216 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ee459213-237c-404d-aa9d-8f681ee6e595-service-ca-bundle") pod "router-default-69f7b748fb-vt9zm" (UID: "ee459213-237c-404d-aa9d-8f681ee6e595") : configmap references non-existent config key: service-ca.crt Aug 04 09:41:09.903660 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:09.903623 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5cce2515-bbb6-4fc5-b6ee-9825796843f2-metrics-tls\") pod \"dns-default-cmck8\" (UID: \"5cce2515-bbb6-4fc5-b6ee-9825796843f2\") " pod="openshift-dns/dns-default-cmck8" Aug 04 09:41:09.903871 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:09.903739 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 09:41:09.903871 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:09.903813 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cce2515-bbb6-4fc5-b6ee-9825796843f2-metrics-tls podName:5cce2515-bbb6-4fc5-b6ee-9825796843f2 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:13.903782098 +0000 UTC m=+42.647916512 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5cce2515-bbb6-4fc5-b6ee-9825796843f2-metrics-tls") pod "dns-default-cmck8" (UID: "5cce2515-bbb6-4fc5-b6ee-9825796843f2") : secret "dns-default-metrics-tls" not found Aug 04 09:41:10.004776 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:10.004738 2572 generic.go:356] "Generic (PLEG): container finished" podID="ec918bd2-ac7e-493e-b164-a16ca18c9422" containerID="c1a23d73d0b5835842ecc257dedd52f3ce1d89a7130b1f7ad8254e6bc7999149" exitCode=0 Aug 04 09:41:10.005194 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:10.004836 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2jnfw" event={"ID":"ec918bd2-ac7e-493e-b164-a16ca18c9422","Type":"ContainerDied","Data":"c1a23d73d0b5835842ecc257dedd52f3ce1d89a7130b1f7ad8254e6bc7999149"} Aug 04 09:41:10.556339 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:10.556169 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:10.657354 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:10.657324 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:11.008713 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:11.008676 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-kptzz" event={"ID":"ef014102-c117-4458-8b5c-169ae01b6d70","Type":"ContainerStarted","Data":"907f92874edf0d792abc752c1f00eb4329cdaf4ac57369e7f9c70a070fdcde08"} Aug 04 09:41:11.010183 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:11.010157 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z68g5" event={"ID":"3d4ecc98-460d-41bb-9fb6-fcea7a43d789","Type":"ContainerStarted","Data":"816d02c925617ab0427fc1da20418ac8d6b08432bd5da0a26af9bd0e87da25a5"} Aug 04 09:41:11.013652 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:11.013628 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2jnfw" event={"ID":"ec918bd2-ac7e-493e-b164-a16ca18c9422","Type":"ContainerStarted","Data":"8dc10c9f60ee20148a650f949bf875b4da1aec711e7f225241481373f10fd86c"} Aug 04 09:41:11.026348 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:11.026300 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-kptzz" podStartSLOduration=1.918202001 podStartE2EDuration="4.0262836s" totalImagesPullingTime="2.108081599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:11.025599586 +0000 UTC m=+39.769734005" watchObservedRunningTime="2026-08-04 09:41:11.0262836 +0000 UTC m=+39.770418001" Aug 04 09:41:11.046458 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:11.046407 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-z68g5" podStartSLOduration=2.755966813 podStartE2EDuration="5.046391151s" totalImagesPullingTime="2.290424338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:41:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:11.045592602 +0000 UTC m=+39.789727024" watchObservedRunningTime="2026-08-04 09:41:11.046391151 +0000 UTC m=+39.790525608" Aug 04 09:41:11.074806 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:11.074742 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2jnfw" podStartSLOduration=8.170979111 podStartE2EDuration="39.074726052s" totalImagesPullingTime="30.903746941s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:40:32 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:11.072223318 +0000 UTC m=+39.816357737" watchObservedRunningTime="2026-08-04 09:41:11.074726052 +0000 UTC m=+39.818860472" Aug 04 09:41:11.138252 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:11.138216 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:11.208878 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:11.208847 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:11.214616 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:11.214591 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee459213-237c-404d-aa9d-8f681ee6e595-metrics-certs\") pod \"router-default-69f7b748fb-vt9zm\" (UID: \"ee459213-237c-404d-aa9d-8f681ee6e595\") " pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:11.214733 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:11.214661 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee459213-237c-404d-aa9d-8f681ee6e595-service-ca-bundle\") pod \"router-default-69f7b748fb-vt9zm\" (UID: \"ee459213-237c-404d-aa9d-8f681ee6e595\") " pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:11.214785 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:11.214766 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Aug 04 09:41:11.214857 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:11.214773 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ee459213-237c-404d-aa9d-8f681ee6e595-service-ca-bundle podName:ee459213-237c-404d-aa9d-8f681ee6e595 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:15.214755515 +0000 UTC m=+43.958889917 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ee459213-237c-404d-aa9d-8f681ee6e595-service-ca-bundle") pod "router-default-69f7b748fb-vt9zm" (UID: "ee459213-237c-404d-aa9d-8f681ee6e595") : configmap references non-existent config key: service-ca.crt Aug 04 09:41:11.214908 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:11.214859 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee459213-237c-404d-aa9d-8f681ee6e595-metrics-certs podName:ee459213-237c-404d-aa9d-8f681ee6e595 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:15.214840998 +0000 UTC m=+43.958975403 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ee459213-237c-404d-aa9d-8f681ee6e595-metrics-certs") pod "router-default-69f7b748fb-vt9zm" (UID: "ee459213-237c-404d-aa9d-8f681ee6e595") : secret "router-metrics-certs-default" not found Aug 04 09:41:11.327533 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:11.327441 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:11.426526 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:11.426487 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:12.713167 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:12.713128 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:13.279540 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:13.279509 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nn7rk"] Aug 04 09:41:13.310939 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:13.310905 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nn7rk"] Aug 04 09:41:13.311077 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:13.311026 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nn7rk" Aug 04 09:41:13.317807 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:13.317769 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Aug 04 09:41:13.318064 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:13.318044 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Aug 04 09:41:13.318154 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:13.318103 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-54lnd\"" Aug 04 09:41:13.318760 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:13.318741 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Aug 04 09:41:13.318857 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:13.318748 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Aug 04 09:41:13.367952 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:13.367922 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:13.432236 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:13.432198 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-data-volume\") pod \"insights-runtime-extractor-nn7rk\" (UID: \"11e66e79-a3c2-4c7c-a283-0fc4ecffd574\") " pod="openshift-insights/insights-runtime-extractor-nn7rk" Aug 04 09:41:13.432406 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:13.432255 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cchcc\" (UniqueName: \"kubernetes.io/projected/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-kube-api-access-cchcc\") pod \"insights-runtime-extractor-nn7rk\" (UID: \"11e66e79-a3c2-4c7c-a283-0fc4ecffd574\") " pod="openshift-insights/insights-runtime-extractor-nn7rk" Aug 04 09:41:13.432406 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:13.432342 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-crio-socket\") pod \"insights-runtime-extractor-nn7rk\" (UID: \"11e66e79-a3c2-4c7c-a283-0fc4ecffd574\") " pod="openshift-insights/insights-runtime-extractor-nn7rk" Aug 04 09:41:13.432406 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:13.432358 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nn7rk\" (UID: \"11e66e79-a3c2-4c7c-a283-0fc4ecffd574\") " pod="openshift-insights/insights-runtime-extractor-nn7rk" Aug 04 09:41:13.432406 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:13.432378 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nn7rk\" (UID: \"11e66e79-a3c2-4c7c-a283-0fc4ecffd574\") " pod="openshift-insights/insights-runtime-extractor-nn7rk" Aug 04 09:41:13.500679 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:13.500644 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:13.533791 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:13.533707 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-crio-socket\") pod \"insights-runtime-extractor-nn7rk\" (UID: \"11e66e79-a3c2-4c7c-a283-0fc4ecffd574\") " pod="openshift-insights/insights-runtime-extractor-nn7rk" Aug 04 09:41:13.533791 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:13.533775 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nn7rk\" (UID: \"11e66e79-a3c2-4c7c-a283-0fc4ecffd574\") " pod="openshift-insights/insights-runtime-extractor-nn7rk" Aug 04 09:41:13.533986 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:13.533812 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nn7rk\" (UID: \"11e66e79-a3c2-4c7c-a283-0fc4ecffd574\") " pod="openshift-insights/insights-runtime-extractor-nn7rk" Aug 04 09:41:13.533986 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:13.533876 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 09:41:13.533986 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:13.533929 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 09:41:13.533986 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:13.533967 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-crio-socket\") pod \"insights-runtime-extractor-nn7rk\" (UID: \"11e66e79-a3c2-4c7c-a283-0fc4ecffd574\") " pod="openshift-insights/insights-runtime-extractor-nn7rk" Aug 04 09:41:13.534105 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:13.533997 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-insights-runtime-extractor-tls podName:11e66e79-a3c2-4c7c-a283-0fc4ecffd574 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:14.033979091 +0000 UTC m=+42.778113489 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nn7rk" (UID: "11e66e79-a3c2-4c7c-a283-0fc4ecffd574") : secret "insights-runtime-extractor-tls" not found Aug 04 09:41:13.534105 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:13.534035 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-kube-rbac-proxy-cm podName:11e66e79-a3c2-4c7c-a283-0fc4ecffd574 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:14.034027959 +0000 UTC m=+42.778162356 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nn7rk" (UID: "11e66e79-a3c2-4c7c-a283-0fc4ecffd574") : configmap "kube-rbac-proxy" not found Aug 04 09:41:13.534105 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:13.534050 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-data-volume\") pod \"insights-runtime-extractor-nn7rk\" (UID: \"11e66e79-a3c2-4c7c-a283-0fc4ecffd574\") " pod="openshift-insights/insights-runtime-extractor-nn7rk" Aug 04 09:41:13.534105 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:13.534088 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cchcc\" (UniqueName: \"kubernetes.io/projected/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-kube-api-access-cchcc\") pod \"insights-runtime-extractor-nn7rk\" (UID: \"11e66e79-a3c2-4c7c-a283-0fc4ecffd574\") " pod="openshift-insights/insights-runtime-extractor-nn7rk" Aug 04 09:41:13.534419 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:13.534401 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-data-volume\") pod \"insights-runtime-extractor-nn7rk\" (UID: \"11e66e79-a3c2-4c7c-a283-0fc4ecffd574\") " pod="openshift-insights/insights-runtime-extractor-nn7rk" Aug 04 09:41:13.548656 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:13.548623 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cchcc\" (UniqueName: \"kubernetes.io/projected/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-kube-api-access-cchcc\") pod \"insights-runtime-extractor-nn7rk\" (UID: \"11e66e79-a3c2-4c7c-a283-0fc4ecffd574\") " pod="openshift-insights/insights-runtime-extractor-nn7rk" Aug 04 09:41:13.936498 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:13.936403 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5cce2515-bbb6-4fc5-b6ee-9825796843f2-metrics-tls\") pod \"dns-default-cmck8\" (UID: \"5cce2515-bbb6-4fc5-b6ee-9825796843f2\") " pod="openshift-dns/dns-default-cmck8" Aug 04 09:41:13.936886 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:13.936551 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 09:41:13.936886 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:13.936626 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cce2515-bbb6-4fc5-b6ee-9825796843f2-metrics-tls podName:5cce2515-bbb6-4fc5-b6ee-9825796843f2 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:21.936607407 +0000 UTC m=+50.680741812 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5cce2515-bbb6-4fc5-b6ee-9825796843f2-metrics-tls") pod "dns-default-cmck8" (UID: "5cce2515-bbb6-4fc5-b6ee-9825796843f2") : secret "dns-default-metrics-tls" not found Aug 04 09:41:14.037379 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:14.037346 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nn7rk\" (UID: \"11e66e79-a3c2-4c7c-a283-0fc4ecffd574\") " pod="openshift-insights/insights-runtime-extractor-nn7rk" Aug 04 09:41:14.037379 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:14.037384 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nn7rk\" (UID: \"11e66e79-a3c2-4c7c-a283-0fc4ecffd574\") " pod="openshift-insights/insights-runtime-extractor-nn7rk" Aug 04 09:41:14.037548 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:14.037481 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 09:41:14.037548 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:14.037491 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 09:41:14.037548 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:14.037538 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-kube-rbac-proxy-cm podName:11e66e79-a3c2-4c7c-a283-0fc4ecffd574 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:15.037523379 +0000 UTC m=+43.781657779 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nn7rk" (UID: "11e66e79-a3c2-4c7c-a283-0fc4ecffd574") : configmap "kube-rbac-proxy" not found Aug 04 09:41:14.037658 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:14.037552 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-insights-runtime-extractor-tls podName:11e66e79-a3c2-4c7c-a283-0fc4ecffd574 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:15.037546103 +0000 UTC m=+43.781680499 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nn7rk" (UID: "11e66e79-a3c2-4c7c-a283-0fc4ecffd574") : secret "insights-runtime-extractor-tls" not found Aug 04 09:41:14.523292 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:14.523257 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:14.616312 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:14.616275 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:14.682623 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:14.682592 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:15.044554 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:15.044515 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nn7rk\" (UID: \"11e66e79-a3c2-4c7c-a283-0fc4ecffd574\") " pod="openshift-insights/insights-runtime-extractor-nn7rk" Aug 04 09:41:15.044554 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:15.044559 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nn7rk\" (UID: \"11e66e79-a3c2-4c7c-a283-0fc4ecffd574\") " pod="openshift-insights/insights-runtime-extractor-nn7rk" Aug 04 09:41:15.044997 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:15.044683 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 09:41:15.044997 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:15.044712 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 09:41:15.044997 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:15.044754 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-insights-runtime-extractor-tls podName:11e66e79-a3c2-4c7c-a283-0fc4ecffd574 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:17.044734945 +0000 UTC m=+45.788869363 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nn7rk" (UID: "11e66e79-a3c2-4c7c-a283-0fc4ecffd574") : secret "insights-runtime-extractor-tls" not found Aug 04 09:41:15.044997 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:15.044769 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-kube-rbac-proxy-cm podName:11e66e79-a3c2-4c7c-a283-0fc4ecffd574 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:17.044761978 +0000 UTC m=+45.788896375 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nn7rk" (UID: "11e66e79-a3c2-4c7c-a283-0fc4ecffd574") : configmap "kube-rbac-proxy" not found Aug 04 09:41:15.170784 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:15.170760 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qvwd8_fa0ec5ee-8ecd-4f22-ae99-cfbe2855395c/dns-node-resolver/0.log" Aug 04 09:41:15.246144 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:15.246110 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee459213-237c-404d-aa9d-8f681ee6e595-metrics-certs\") pod \"router-default-69f7b748fb-vt9zm\" (UID: \"ee459213-237c-404d-aa9d-8f681ee6e595\") " pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:15.246273 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:15.246258 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Aug 04 09:41:15.246317 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:15.246301 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee459213-237c-404d-aa9d-8f681ee6e595-service-ca-bundle\") pod \"router-default-69f7b748fb-vt9zm\" (UID: \"ee459213-237c-404d-aa9d-8f681ee6e595\") " pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:15.246349 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:15.246311 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee459213-237c-404d-aa9d-8f681ee6e595-metrics-certs podName:ee459213-237c-404d-aa9d-8f681ee6e595 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:23.246297358 +0000 UTC m=+51.990431782 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ee459213-237c-404d-aa9d-8f681ee6e595-metrics-certs") pod "router-default-69f7b748fb-vt9zm" (UID: "ee459213-237c-404d-aa9d-8f681ee6e595") : secret "router-metrics-certs-default" not found Aug 04 09:41:15.246423 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:15.246413 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ee459213-237c-404d-aa9d-8f681ee6e595-service-ca-bundle podName:ee459213-237c-404d-aa9d-8f681ee6e595 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:23.246400843 +0000 UTC m=+51.990535241 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ee459213-237c-404d-aa9d-8f681ee6e595-service-ca-bundle") pod "router-default-69f7b748fb-vt9zm" (UID: "ee459213-237c-404d-aa9d-8f681ee6e595") : configmap references non-existent config key: service-ca.crt Aug 04 09:41:15.769855 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:15.769828 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-57rd7_a9ba7a09-0ff2-4657-8bbd-47b0af96a45a/node-ca/0.log" Aug 04 09:41:16.951177 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:16.951143 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:16.969811 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:16.969781 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z68g5_3d4ecc98-460d-41bb-9fb6-fcea7a43d789/serve-healthcheck-canary/0.log" Aug 04 09:41:17.055010 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:17.054981 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:17.061119 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:17.061086 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nn7rk\" (UID: \"11e66e79-a3c2-4c7c-a283-0fc4ecffd574\") " pod="openshift-insights/insights-runtime-extractor-nn7rk" Aug 04 09:41:17.061282 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:17.061125 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nn7rk\" (UID: \"11e66e79-a3c2-4c7c-a283-0fc4ecffd574\") " pod="openshift-insights/insights-runtime-extractor-nn7rk" Aug 04 09:41:17.061282 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:17.061236 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 09:41:17.061282 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:17.061246 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 09:41:17.061404 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:17.061295 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-insights-runtime-extractor-tls podName:11e66e79-a3c2-4c7c-a283-0fc4ecffd574 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:21.061278847 +0000 UTC m=+49.805413245 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nn7rk" (UID: "11e66e79-a3c2-4c7c-a283-0fc4ecffd574") : secret "insights-runtime-extractor-tls" not found Aug 04 09:41:17.061404 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:17.061309 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-kube-rbac-proxy-cm podName:11e66e79-a3c2-4c7c-a283-0fc4ecffd574 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:21.061302231 +0000 UTC m=+49.805436628 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nn7rk" (UID: "11e66e79-a3c2-4c7c-a283-0fc4ecffd574") : configmap "kube-rbac-proxy" not found Aug 04 09:41:17.242028 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:17.241916 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:17.369027 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:17.368995 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:17.761811 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:17.761765 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:17.860720 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:17.860686 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:18.015074 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:18.014992 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:18.123129 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:18.123092 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:18.366680 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:18.366596 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:18.461887 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:18.461860 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:19.239973 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:19.239941 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-xdqgl"] Aug 04 09:41:19.272762 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:19.272724 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:19.273468 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:19.273452 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-xdqgl"] Aug 04 09:41:19.273553 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:19.273542 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-xdqgl" Aug 04 09:41:19.276676 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:19.276653 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-rpljh\"" Aug 04 09:41:19.277017 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:19.276998 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Aug 04 09:41:19.277940 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:19.277920 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Aug 04 09:41:19.278047 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:19.277968 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Aug 04 09:41:19.278047 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:19.277988 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Aug 04 09:41:19.279392 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:19.279364 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e101a4e3-c0c2-4763-b9c3-a5d98324d0c7-signing-key\") pod \"service-ca-98c88fbb5-xdqgl\" (UID: \"e101a4e3-c0c2-4763-b9c3-a5d98324d0c7\") " pod="openshift-service-ca/service-ca-98c88fbb5-xdqgl" Aug 04 09:41:19.279494 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:19.279409 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e101a4e3-c0c2-4763-b9c3-a5d98324d0c7-signing-cabundle\") pod \"service-ca-98c88fbb5-xdqgl\" (UID: \"e101a4e3-c0c2-4763-b9c3-a5d98324d0c7\") " pod="openshift-service-ca/service-ca-98c88fbb5-xdqgl" Aug 04 09:41:19.279918 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:19.279605 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpsl5\" (UniqueName: \"kubernetes.io/projected/e101a4e3-c0c2-4763-b9c3-a5d98324d0c7-kube-api-access-vpsl5\") pod \"service-ca-98c88fbb5-xdqgl\" (UID: \"e101a4e3-c0c2-4763-b9c3-a5d98324d0c7\") " pod="openshift-service-ca/service-ca-98c88fbb5-xdqgl" Aug 04 09:41:19.347457 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:19.347421 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:19.380358 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:19.380320 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpsl5\" (UniqueName: \"kubernetes.io/projected/e101a4e3-c0c2-4763-b9c3-a5d98324d0c7-kube-api-access-vpsl5\") pod \"service-ca-98c88fbb5-xdqgl\" (UID: \"e101a4e3-c0c2-4763-b9c3-a5d98324d0c7\") " pod="openshift-service-ca/service-ca-98c88fbb5-xdqgl" Aug 04 09:41:19.380503 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:19.380392 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e101a4e3-c0c2-4763-b9c3-a5d98324d0c7-signing-key\") pod \"service-ca-98c88fbb5-xdqgl\" (UID: \"e101a4e3-c0c2-4763-b9c3-a5d98324d0c7\") " pod="openshift-service-ca/service-ca-98c88fbb5-xdqgl" Aug 04 09:41:19.380503 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:19.380410 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e101a4e3-c0c2-4763-b9c3-a5d98324d0c7-signing-cabundle\") pod \"service-ca-98c88fbb5-xdqgl\" (UID: \"e101a4e3-c0c2-4763-b9c3-a5d98324d0c7\") " pod="openshift-service-ca/service-ca-98c88fbb5-xdqgl" Aug 04 09:41:19.381089 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:19.381072 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e101a4e3-c0c2-4763-b9c3-a5d98324d0c7-signing-cabundle\") pod \"service-ca-98c88fbb5-xdqgl\" (UID: \"e101a4e3-c0c2-4763-b9c3-a5d98324d0c7\") " pod="openshift-service-ca/service-ca-98c88fbb5-xdqgl" Aug 04 09:41:19.383060 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:19.383042 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e101a4e3-c0c2-4763-b9c3-a5d98324d0c7-signing-key\") pod \"service-ca-98c88fbb5-xdqgl\" (UID: \"e101a4e3-c0c2-4763-b9c3-a5d98324d0c7\") " pod="openshift-service-ca/service-ca-98c88fbb5-xdqgl" Aug 04 09:41:19.388972 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:19.388949 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpsl5\" (UniqueName: \"kubernetes.io/projected/e101a4e3-c0c2-4763-b9c3-a5d98324d0c7-kube-api-access-vpsl5\") pod \"service-ca-98c88fbb5-xdqgl\" (UID: \"e101a4e3-c0c2-4763-b9c3-a5d98324d0c7\") " pod="openshift-service-ca/service-ca-98c88fbb5-xdqgl" Aug 04 09:41:19.582605 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:19.582520 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-xdqgl" Aug 04 09:41:19.725468 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:19.725340 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-xdqgl"] Aug 04 09:41:19.727943 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:41:19.727910 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode101a4e3_c0c2_4763_b9c3_a5d98324d0c7.slice/crio-bb7e7c47976daa5c63b23ca1240a470a00aa603f2e6bbc9a8689d12f371cd4c5 WatchSource:0}: Error finding container bb7e7c47976daa5c63b23ca1240a470a00aa603f2e6bbc9a8689d12f371cd4c5: Status 404 returned error can't find the container with id bb7e7c47976daa5c63b23ca1240a470a00aa603f2e6bbc9a8689d12f371cd4c5 Aug 04 09:41:20.035921 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:20.035877 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-xdqgl" event={"ID":"e101a4e3-c0c2-4763-b9c3-a5d98324d0c7","Type":"ContainerStarted","Data":"bb7e7c47976daa5c63b23ca1240a470a00aa603f2e6bbc9a8689d12f371cd4c5"} Aug 04 09:41:20.365812 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:20.365714 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:21.094609 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:21.094565 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nn7rk\" (UID: \"11e66e79-a3c2-4c7c-a283-0fc4ecffd574\") " pod="openshift-insights/insights-runtime-extractor-nn7rk" Aug 04 09:41:21.094820 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:21.094629 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nn7rk\" (UID: \"11e66e79-a3c2-4c7c-a283-0fc4ecffd574\") " pod="openshift-insights/insights-runtime-extractor-nn7rk" Aug 04 09:41:21.094820 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:21.094696 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 09:41:21.094820 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:21.094769 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 09:41:21.094820 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:21.094779 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-insights-runtime-extractor-tls podName:11e66e79-a3c2-4c7c-a283-0fc4ecffd574 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:29.094757852 +0000 UTC m=+57.838892267 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nn7rk" (UID: "11e66e79-a3c2-4c7c-a283-0fc4ecffd574") : secret "insights-runtime-extractor-tls" not found Aug 04 09:41:21.095017 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:21.094847 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-kube-rbac-proxy-cm podName:11e66e79-a3c2-4c7c-a283-0fc4ecffd574 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:29.094829606 +0000 UTC m=+57.838964009 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nn7rk" (UID: "11e66e79-a3c2-4c7c-a283-0fc4ecffd574") : configmap "kube-rbac-proxy" not found Aug 04 09:41:22.002757 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:22.002713 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5cce2515-bbb6-4fc5-b6ee-9825796843f2-metrics-tls\") pod \"dns-default-cmck8\" (UID: \"5cce2515-bbb6-4fc5-b6ee-9825796843f2\") " pod="openshift-dns/dns-default-cmck8" Aug 04 09:41:22.003237 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:22.002846 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 09:41:22.003237 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:22.002900 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cce2515-bbb6-4fc5-b6ee-9825796843f2-metrics-tls podName:5cce2515-bbb6-4fc5-b6ee-9825796843f2 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:38.002885707 +0000 UTC m=+66.747020117 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5cce2515-bbb6-4fc5-b6ee-9825796843f2-metrics-tls") pod "dns-default-cmck8" (UID: "5cce2515-bbb6-4fc5-b6ee-9825796843f2") : secret "dns-default-metrics-tls" not found Aug 04 09:41:22.041761 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:22.041671 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-xdqgl" event={"ID":"e101a4e3-c0c2-4763-b9c3-a5d98324d0c7","Type":"ContainerStarted","Data":"cff0342cd12610d41d124cac2697195c688421590884db40ab1dd16bd8c5d792"} Aug 04 09:41:22.063324 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:22.063261 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-98c88fbb5-xdqgl" podStartSLOduration=1.013211368 podStartE2EDuration="3.063240322s" totalImagesPullingTime="2.050028954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:41:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:22.062524426 +0000 UTC m=+50.806658847" watchObservedRunningTime="2026-08-04 09:41:22.063240322 +0000 UTC m=+50.807374744" Aug 04 09:41:23.312744 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:23.312702 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee459213-237c-404d-aa9d-8f681ee6e595-service-ca-bundle\") pod \"router-default-69f7b748fb-vt9zm\" (UID: \"ee459213-237c-404d-aa9d-8f681ee6e595\") " pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:23.313137 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:23.312770 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee459213-237c-404d-aa9d-8f681ee6e595-metrics-certs\") pod \"router-default-69f7b748fb-vt9zm\" (UID: \"ee459213-237c-404d-aa9d-8f681ee6e595\") " pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:23.313137 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:23.312899 2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Aug 04 09:41:23.313137 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:23.312916 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ee459213-237c-404d-aa9d-8f681ee6e595-service-ca-bundle podName:ee459213-237c-404d-aa9d-8f681ee6e595 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:39.312896034 +0000 UTC m=+68.057030444 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ee459213-237c-404d-aa9d-8f681ee6e595-service-ca-bundle") pod "router-default-69f7b748fb-vt9zm" (UID: "ee459213-237c-404d-aa9d-8f681ee6e595") : configmap references non-existent config key: service-ca.crt Aug 04 09:41:23.313137 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:23.312945 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee459213-237c-404d-aa9d-8f681ee6e595-metrics-certs podName:ee459213-237c-404d-aa9d-8f681ee6e595 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:39.312934716 +0000 UTC m=+68.057069128 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ee459213-237c-404d-aa9d-8f681ee6e595-metrics-certs") pod "router-default-69f7b748fb-vt9zm" (UID: "ee459213-237c-404d-aa9d-8f681ee6e595") : secret "router-metrics-certs-default" not found Aug 04 09:41:23.388571 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:23.388497 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:23.689296 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:23.689204 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:23.833641 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:23.833581 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:23.933207 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:23.933168 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:24.207224 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:24.207174 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:24.712317 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:24.712283 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:24.916260 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:24.916228 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:25.303209 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:25.303174 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:25.698088 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:25.698052 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:25.890110 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:25.890045 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:26.304286 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:26.304253 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:26.404150 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:26.404114 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:26.904808 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:26.904757 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:27.127376 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:27.127345 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:27.238687 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:27.238656 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:27.521389 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:27.521312 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:27.690100 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:27.690064 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:28.310415 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:28.310380 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:28.489367 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:28.489332 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:28.684194 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:28.684107 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:28.896091 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:28.896052 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:28.973344 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:28.973312 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:29.160230 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:29.160192 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nn7rk\" (UID: \"11e66e79-a3c2-4c7c-a283-0fc4ecffd574\") " pod="openshift-insights/insights-runtime-extractor-nn7rk" Aug 04 09:41:29.160230 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:29.160229 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nn7rk\" (UID: \"11e66e79-a3c2-4c7c-a283-0fc4ecffd574\") " pod="openshift-insights/insights-runtime-extractor-nn7rk" Aug 04 09:41:29.160464 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:29.160353 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 09:41:29.160464 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:29.160425 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-insights-runtime-extractor-tls podName:11e66e79-a3c2-4c7c-a283-0fc4ecffd574 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:45.160406508 +0000 UTC m=+73.904540920 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nn7rk" (UID: "11e66e79-a3c2-4c7c-a283-0fc4ecffd574") : secret "insights-runtime-extractor-tls" not found Aug 04 09:41:29.160698 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:29.160682 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nn7rk\" (UID: \"11e66e79-a3c2-4c7c-a283-0fc4ecffd574\") " pod="openshift-insights/insights-runtime-extractor-nn7rk" Aug 04 09:41:29.560742 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:29.560696 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:29.886530 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:29.886450 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:30.316201 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:30.316166 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:30.510892 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:30.510856 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:30.736598 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:30.736560 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:30.925348 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:30.925293 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:32.111148 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:32.111076 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:32.540485 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:32.540449 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:32.648110 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:32.648061 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:32.733073 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:32.733040 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:32.912786 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:32.912687 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:32.977141 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:32.977114 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8sdsr" Aug 04 09:41:33.733180 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:33.733134 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:33.901436 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:33.901402 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:34.100245 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:34.100159 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:35.758469 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:35.758423 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:35.831890 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:35.831853 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:36.417631 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:36.417578 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f732a38-dafb-42d2-acc6-ff8059c65350-original-pull-secret\") pod \"global-pull-secret-syncer-7wl64\" (UID: \"8f732a38-dafb-42d2-acc6-ff8059c65350\") " pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:41:36.420558 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:36.420536 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Aug 04 09:41:36.429930 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:36.429907 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f732a38-dafb-42d2-acc6-ff8059c65350-original-pull-secret\") pod \"global-pull-secret-syncer-7wl64\" (UID: \"8f732a38-dafb-42d2-acc6-ff8059c65350\") " pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:41:36.518487 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:36.518435 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7e0b230-f265-4c9b-9d1d-9008457d8aa1-metrics-certs\") pod \"network-metrics-daemon-pjdn8\" (UID: \"b7e0b230-f265-4c9b-9d1d-9008457d8aa1\") " pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:41:36.521533 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:36.521513 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Aug 04 09:41:36.531263 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:36.531226 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b7e0b230-f265-4c9b-9d1d-9008457d8aa1-metrics-certs\") pod \"network-metrics-daemon-pjdn8\" (UID: \"b7e0b230-f265-4c9b-9d1d-9008457d8aa1\") " pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:41:36.592379 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:36.592343 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7wl64" Aug 04 09:41:36.602113 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:36.602086 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zpz8l\"" Aug 04 09:41:36.609740 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:36.609708 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pjdn8" Aug 04 09:41:36.619103 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:36.619071 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q78c\" (UniqueName: \"kubernetes.io/projected/6cc7e819-7680-4323-8afa-afa00c9433ce-kube-api-access-4q78c\") pod \"network-check-target-qcql5\" (UID: \"6cc7e819-7680-4323-8afa-afa00c9433ce\") " pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:41:36.622309 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:36.622287 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Aug 04 09:41:36.632910 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:36.632879 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Aug 04 09:41:36.642592 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:36.642561 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q78c\" (UniqueName: \"kubernetes.io/projected/6cc7e819-7680-4323-8afa-afa00c9433ce-kube-api-access-4q78c\") pod \"network-check-target-qcql5\" (UID: \"6cc7e819-7680-4323-8afa-afa00c9433ce\") " pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:41:36.739750 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:36.739719 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7wl64"] Aug 04 09:41:36.740714 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:41:36.740690 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f732a38_dafb_42d2_acc6_ff8059c65350.slice/crio-1b7a7ae1ce39c28dad546e08cfbd816024fae91a9eda708a495b65879a7119c2 WatchSource:0}: Error finding container 1b7a7ae1ce39c28dad546e08cfbd816024fae91a9eda708a495b65879a7119c2: Status 404 returned error can't find the container with id 1b7a7ae1ce39c28dad546e08cfbd816024fae91a9eda708a495b65879a7119c2 Aug 04 09:41:36.744065 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:36.744040 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pjdn8"] Aug 04 09:41:36.746837 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:41:36.746811 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7e0b230_f265_4c9b_9d1d_9008457d8aa1.slice/crio-8bf85d3bca55942f397ac73575b0224c8c4ecb85d50040788898c333e27c35ad WatchSource:0}: Error finding container 8bf85d3bca55942f397ac73575b0224c8c4ecb85d50040788898c333e27c35ad: Status 404 returned error can't find the container with id 8bf85d3bca55942f397ac73575b0224c8c4ecb85d50040788898c333e27c35ad Aug 04 09:41:36.887952 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:36.887918 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gfsnh\"" Aug 04 09:41:36.895512 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:36.895490 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:41:37.023411 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:37.023380 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qcql5"] Aug 04 09:41:37.026948 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:41:37.026918 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cc7e819_7680_4323_8afa_afa00c9433ce.slice/crio-253e4fbf7e84cff8b806ff66b724c269cfe7947df5e25ae7ea18d043036a8efc WatchSource:0}: Error finding container 253e4fbf7e84cff8b806ff66b724c269cfe7947df5e25ae7ea18d043036a8efc: Status 404 returned error can't find the container with id 253e4fbf7e84cff8b806ff66b724c269cfe7947df5e25ae7ea18d043036a8efc Aug 04 09:41:37.081509 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:37.081464 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7wl64" event={"ID":"8f732a38-dafb-42d2-acc6-ff8059c65350","Type":"ContainerStarted","Data":"1b7a7ae1ce39c28dad546e08cfbd816024fae91a9eda708a495b65879a7119c2"} Aug 04 09:41:37.082976 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:37.082949 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qcql5" event={"ID":"6cc7e819-7680-4323-8afa-afa00c9433ce","Type":"ContainerStarted","Data":"253e4fbf7e84cff8b806ff66b724c269cfe7947df5e25ae7ea18d043036a8efc"} Aug 04 09:41:37.084526 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:37.084491 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pjdn8" event={"ID":"b7e0b230-f265-4c9b-9d1d-9008457d8aa1","Type":"ContainerStarted","Data":"8bf85d3bca55942f397ac73575b0224c8c4ecb85d50040788898c333e27c35ad"} Aug 04 09:41:38.031581 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:38.031538 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5cce2515-bbb6-4fc5-b6ee-9825796843f2-metrics-tls\") pod \"dns-default-cmck8\" (UID: \"5cce2515-bbb6-4fc5-b6ee-9825796843f2\") " pod="openshift-dns/dns-default-cmck8" Aug 04 09:41:38.035332 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:38.035266 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5cce2515-bbb6-4fc5-b6ee-9825796843f2-metrics-tls\") pod \"dns-default-cmck8\" (UID: \"5cce2515-bbb6-4fc5-b6ee-9825796843f2\") " pod="openshift-dns/dns-default-cmck8" Aug 04 09:41:38.092588 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:38.092532 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pjdn8" event={"ID":"b7e0b230-f265-4c9b-9d1d-9008457d8aa1","Type":"ContainerStarted","Data":"26da34faab82200ba58aa822667c04ee2711e90d308b9886f48ce177df1749e5"} Aug 04 09:41:38.258861 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:38.258775 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-psh6m\"" Aug 04 09:41:38.266488 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:38.266455 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cmck8" Aug 04 09:41:38.422846 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:38.422694 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cmck8"] Aug 04 09:41:38.425902 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:41:38.425864 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cce2515_bbb6_4fc5_b6ee_9825796843f2.slice/crio-299bd6f479889e0785aefe3bb269b7545a8ac2e9117abfca2ab4e39615a07755 WatchSource:0}: Error finding container 299bd6f479889e0785aefe3bb269b7545a8ac2e9117abfca2ab4e39615a07755: Status 404 returned error can't find the container with id 299bd6f479889e0785aefe3bb269b7545a8ac2e9117abfca2ab4e39615a07755 Aug 04 09:41:39.105997 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:39.105442 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pjdn8" event={"ID":"b7e0b230-f265-4c9b-9d1d-9008457d8aa1","Type":"ContainerStarted","Data":"f348a3e1adc4c25e537b58d935ce9202579fa9ea2555fe09a1699650d5e37214"} Aug 04 09:41:39.107696 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:39.107642 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cmck8" event={"ID":"5cce2515-bbb6-4fc5-b6ee-9825796843f2","Type":"ContainerStarted","Data":"299bd6f479889e0785aefe3bb269b7545a8ac2e9117abfca2ab4e39615a07755"} Aug 04 09:41:39.126532 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:39.126467 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-pjdn8" podStartSLOduration=66.007449427 podStartE2EDuration="1m7.12644459s" totalImagesPullingTime="1.118995163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:40:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:39.12596693 +0000 UTC m=+67.870101345" watchObservedRunningTime="2026-08-04 09:41:39.12644459 +0000 UTC m=+67.870579009" Aug 04 09:41:39.342870 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:39.342224 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee459213-237c-404d-aa9d-8f681ee6e595-service-ca-bundle\") pod \"router-default-69f7b748fb-vt9zm\" (UID: \"ee459213-237c-404d-aa9d-8f681ee6e595\") " pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:39.342870 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:39.342309 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee459213-237c-404d-aa9d-8f681ee6e595-metrics-certs\") pod \"router-default-69f7b748fb-vt9zm\" (UID: \"ee459213-237c-404d-aa9d-8f681ee6e595\") " pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:39.343519 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:39.343457 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee459213-237c-404d-aa9d-8f681ee6e595-service-ca-bundle\") pod \"router-default-69f7b748fb-vt9zm\" (UID: \"ee459213-237c-404d-aa9d-8f681ee6e595\") " pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:39.345999 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:39.345937 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee459213-237c-404d-aa9d-8f681ee6e595-metrics-certs\") pod \"router-default-69f7b748fb-vt9zm\" (UID: \"ee459213-237c-404d-aa9d-8f681ee6e595\") " pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:39.596720 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:39.596678 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-95kdr\"" Aug 04 09:41:39.604458 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:39.604428 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:42.056434 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:42.056400 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:42.184547 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:42.184510 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:42.285146 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:42.285109 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:42.474351 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:42.474311 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:42.697708 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:42.697670 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-69f7b748fb-vt9zm"] Aug 04 09:41:43.117514 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.117427 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bff96c9fd-q9gj4"] Aug 04 09:41:43.119436 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.119418 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bff96c9fd-q9gj4" Aug 04 09:41:43.125571 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.125534 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qcql5" event={"ID":"6cc7e819-7680-4323-8afa-afa00c9433ce","Type":"ContainerStarted","Data":"c15c85921d3c9055cddb8d80066acdbef1a754580a2708207720321b4cebf98f"} Aug 04 09:41:43.125732 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.125648 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:41:43.127162 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.127139 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Aug 04 09:41:43.127355 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.127337 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Aug 04 09:41:43.127700 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.127679 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cmck8" event={"ID":"5cce2515-bbb6-4fc5-b6ee-9825796843f2","Type":"ContainerStarted","Data":"84fac8d17c4a1e6f68fb2ac9c57b06d268f8464f526eecfa454847cd63af0e7c"} Aug 04 09:41:43.127789 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.127717 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cmck8" event={"ID":"5cce2515-bbb6-4fc5-b6ee-9825796843f2","Type":"ContainerStarted","Data":"95d0e7b53bd6303b224fb8158490e61906cbbdb7ac629623032dbd1c1df608bf"} Aug 04 09:41:43.127789 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.127738 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cmck8" Aug 04 09:41:43.128420 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.128392 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Aug 04 09:41:43.128584 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.128566 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-vwkdd\"" Aug 04 09:41:43.128658 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.128568 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Aug 04 09:41:43.129384 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.129358 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-69f7b748fb-vt9zm" event={"ID":"ee459213-237c-404d-aa9d-8f681ee6e595","Type":"ContainerStarted","Data":"aee9c66bbeff740157a0d7cfb91f52f5875d7aed60091aa3f18c44d96c20be49"} Aug 04 09:41:43.129492 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.129400 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-69f7b748fb-vt9zm" event={"ID":"ee459213-237c-404d-aa9d-8f681ee6e595","Type":"ContainerStarted","Data":"d01b5d84ba6473680f09ce0aca220a4e348d147cdfbec0e9cd9106a9e8d56998"} Aug 04 09:41:43.130733 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.130708 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7wl64" event={"ID":"8f732a38-dafb-42d2-acc6-ff8059c65350","Type":"ContainerStarted","Data":"8d5bec34396c515341918b60922f14cf7094ff31ac7cf64818cd44fb668869cf"} Aug 04 09:41:43.135551 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.135528 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bff96c9fd-q9gj4"] Aug 04 09:41:43.178134 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.178096 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwbwl\" (UniqueName: \"kubernetes.io/projected/6fe622e3-2407-4db6-977f-6c8d1ece43f8-kube-api-access-lwbwl\") pod \"managed-serviceaccount-addon-agent-bff96c9fd-q9gj4\" (UID: \"6fe622e3-2407-4db6-977f-6c8d1ece43f8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bff96c9fd-q9gj4" Aug 04 09:41:43.178134 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.178137 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6fe622e3-2407-4db6-977f-6c8d1ece43f8-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-bff96c9fd-q9gj4\" (UID: \"6fe622e3-2407-4db6-977f-6c8d1ece43f8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bff96c9fd-q9gj4" Aug 04 09:41:43.186751 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.186707 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qcql5" podStartSLOduration=65.67386591 podStartE2EDuration="1m11.186691379s" totalImagesPullingTime="5.512825469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:40:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:43.184574995 +0000 UTC m=+71.928709427" watchObservedRunningTime="2026-08-04 09:41:43.186691379 +0000 UTC m=+71.930825798" Aug 04 09:41:43.202936 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.202902 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-588c6c96bb-gz8pg"] Aug 04 09:41:43.204757 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.204740 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" Aug 04 09:41:43.209494 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.209473 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-b4dvw\"" Aug 04 09:41:43.210109 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.210080 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Aug 04 09:41:43.211007 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.210988 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Aug 04 09:41:43.212170 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.212155 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Aug 04 09:41:43.220376 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.220355 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Aug 04 09:41:43.222743 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.222722 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:43.226761 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.226736 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-588c6c96bb-gz8pg"] Aug 04 09:41:43.279129 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.279090 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7f40804f-b0dd-4a82-982c-da8591f7fdf5-image-registry-private-configuration\") pod \"image-registry-588c6c96bb-gz8pg\" (UID: \"7f40804f-b0dd-4a82-982c-da8591f7fdf5\") " pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" Aug 04 09:41:43.279298 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.279147 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7f40804f-b0dd-4a82-982c-da8591f7fdf5-registry-certificates\") pod \"image-registry-588c6c96bb-gz8pg\" (UID: \"7f40804f-b0dd-4a82-982c-da8591f7fdf5\") " pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" Aug 04 09:41:43.279298 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.279183 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95zwj\" (UniqueName: \"kubernetes.io/projected/7f40804f-b0dd-4a82-982c-da8591f7fdf5-kube-api-access-95zwj\") pod \"image-registry-588c6c96bb-gz8pg\" (UID: \"7f40804f-b0dd-4a82-982c-da8591f7fdf5\") " pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" Aug 04 09:41:43.279298 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.279250 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7f40804f-b0dd-4a82-982c-da8591f7fdf5-trusted-ca\") pod \"image-registry-588c6c96bb-gz8pg\" (UID: \"7f40804f-b0dd-4a82-982c-da8591f7fdf5\") " pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" Aug 04 09:41:43.279298 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.279271 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7f40804f-b0dd-4a82-982c-da8591f7fdf5-bound-sa-token\") pod \"image-registry-588c6c96bb-gz8pg\" (UID: \"7f40804f-b0dd-4a82-982c-da8591f7fdf5\") " pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" Aug 04 09:41:43.279439 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.279297 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7f40804f-b0dd-4a82-982c-da8591f7fdf5-installation-pull-secrets\") pod \"image-registry-588c6c96bb-gz8pg\" (UID: \"7f40804f-b0dd-4a82-982c-da8591f7fdf5\") " pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" Aug 04 09:41:43.279439 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.279352 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwbwl\" (UniqueName: \"kubernetes.io/projected/6fe622e3-2407-4db6-977f-6c8d1ece43f8-kube-api-access-lwbwl\") pod \"managed-serviceaccount-addon-agent-bff96c9fd-q9gj4\" (UID: \"6fe622e3-2407-4db6-977f-6c8d1ece43f8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bff96c9fd-q9gj4" Aug 04 09:41:43.279439 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.279368 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7f40804f-b0dd-4a82-982c-da8591f7fdf5-registry-tls\") pod \"image-registry-588c6c96bb-gz8pg\" (UID: \"7f40804f-b0dd-4a82-982c-da8591f7fdf5\") " pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" Aug 04 09:41:43.279439 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.279397 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6fe622e3-2407-4db6-977f-6c8d1ece43f8-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-bff96c9fd-q9gj4\" (UID: \"6fe622e3-2407-4db6-977f-6c8d1ece43f8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bff96c9fd-q9gj4" Aug 04 09:41:43.279439 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.279418 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7f40804f-b0dd-4a82-982c-da8591f7fdf5-ca-trust-extracted\") pod \"image-registry-588c6c96bb-gz8pg\" (UID: \"7f40804f-b0dd-4a82-982c-da8591f7fdf5\") " pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" Aug 04 09:41:43.281778 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.281759 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6fe622e3-2407-4db6-977f-6c8d1ece43f8-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-bff96c9fd-q9gj4\" (UID: \"6fe622e3-2407-4db6-977f-6c8d1ece43f8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bff96c9fd-q9gj4" Aug 04 09:41:43.299982 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.299930 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7wl64" podStartSLOduration=66.507743192 podStartE2EDuration="1m12.299909659s" totalImagesPullingTime="5.792166467s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:40:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:43.295985787 +0000 UTC m=+72.040120207" watchObservedRunningTime="2026-08-04 09:41:43.299909659 +0000 UTC m=+72.044044077" Aug 04 09:41:43.300371 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.300350 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-cmck8" podStartSLOduration=33.187198913 podStartE2EDuration="37.300342529s" totalImagesPullingTime="4.113143616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:41:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:43.236647912 +0000 UTC m=+71.980782345" watchObservedRunningTime="2026-08-04 09:41:43.300342529 +0000 UTC m=+72.044476946" Aug 04 09:41:43.316306 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.316271 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwbwl\" (UniqueName: \"kubernetes.io/projected/6fe622e3-2407-4db6-977f-6c8d1ece43f8-kube-api-access-lwbwl\") pod \"managed-serviceaccount-addon-agent-bff96c9fd-q9gj4\" (UID: \"6fe622e3-2407-4db6-977f-6c8d1ece43f8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bff96c9fd-q9gj4" Aug 04 09:41:43.380550 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.380467 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7f40804f-b0dd-4a82-982c-da8591f7fdf5-installation-pull-secrets\") pod \"image-registry-588c6c96bb-gz8pg\" (UID: \"7f40804f-b0dd-4a82-982c-da8591f7fdf5\") " pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" Aug 04 09:41:43.380550 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.380506 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7f40804f-b0dd-4a82-982c-da8591f7fdf5-registry-tls\") pod \"image-registry-588c6c96bb-gz8pg\" (UID: \"7f40804f-b0dd-4a82-982c-da8591f7fdf5\") " pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" Aug 04 09:41:43.380550 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.380528 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7f40804f-b0dd-4a82-982c-da8591f7fdf5-ca-trust-extracted\") pod \"image-registry-588c6c96bb-gz8pg\" (UID: \"7f40804f-b0dd-4a82-982c-da8591f7fdf5\") " pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" Aug 04 09:41:43.380783 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.380659 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7f40804f-b0dd-4a82-982c-da8591f7fdf5-image-registry-private-configuration\") pod \"image-registry-588c6c96bb-gz8pg\" (UID: \"7f40804f-b0dd-4a82-982c-da8591f7fdf5\") " pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" Aug 04 09:41:43.380783 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.380715 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7f40804f-b0dd-4a82-982c-da8591f7fdf5-registry-certificates\") pod \"image-registry-588c6c96bb-gz8pg\" (UID: \"7f40804f-b0dd-4a82-982c-da8591f7fdf5\") " pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" Aug 04 09:41:43.380783 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.380738 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95zwj\" (UniqueName: \"kubernetes.io/projected/7f40804f-b0dd-4a82-982c-da8591f7fdf5-kube-api-access-95zwj\") pod \"image-registry-588c6c96bb-gz8pg\" (UID: \"7f40804f-b0dd-4a82-982c-da8591f7fdf5\") " pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" Aug 04 09:41:43.380783 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.380780 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7f40804f-b0dd-4a82-982c-da8591f7fdf5-trusted-ca\") pod \"image-registry-588c6c96bb-gz8pg\" (UID: \"7f40804f-b0dd-4a82-982c-da8591f7fdf5\") " pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" Aug 04 09:41:43.381016 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.380834 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7f40804f-b0dd-4a82-982c-da8591f7fdf5-bound-sa-token\") pod \"image-registry-588c6c96bb-gz8pg\" (UID: \"7f40804f-b0dd-4a82-982c-da8591f7fdf5\") " pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" Aug 04 09:41:43.381016 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.380940 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7f40804f-b0dd-4a82-982c-da8591f7fdf5-ca-trust-extracted\") pod \"image-registry-588c6c96bb-gz8pg\" (UID: \"7f40804f-b0dd-4a82-982c-da8591f7fdf5\") " pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" Aug 04 09:41:43.381568 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.381542 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7f40804f-b0dd-4a82-982c-da8591f7fdf5-registry-certificates\") pod \"image-registry-588c6c96bb-gz8pg\" (UID: \"7f40804f-b0dd-4a82-982c-da8591f7fdf5\") " pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" Aug 04 09:41:43.381834 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.381789 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7f40804f-b0dd-4a82-982c-da8591f7fdf5-trusted-ca\") pod \"image-registry-588c6c96bb-gz8pg\" (UID: \"7f40804f-b0dd-4a82-982c-da8591f7fdf5\") " pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" Aug 04 09:41:43.383094 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.383072 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7f40804f-b0dd-4a82-982c-da8591f7fdf5-registry-tls\") pod \"image-registry-588c6c96bb-gz8pg\" (UID: \"7f40804f-b0dd-4a82-982c-da8591f7fdf5\") " pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" Aug 04 09:41:43.383191 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.383094 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7f40804f-b0dd-4a82-982c-da8591f7fdf5-installation-pull-secrets\") pod \"image-registry-588c6c96bb-gz8pg\" (UID: \"7f40804f-b0dd-4a82-982c-da8591f7fdf5\") " pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" Aug 04 09:41:43.383191 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.383133 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7f40804f-b0dd-4a82-982c-da8591f7fdf5-image-registry-private-configuration\") pod \"image-registry-588c6c96bb-gz8pg\" (UID: \"7f40804f-b0dd-4a82-982c-da8591f7fdf5\") " pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" Aug 04 09:41:43.388182 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.388141 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-69f7b748fb-vt9zm" podStartSLOduration=36.388126853 podStartE2EDuration="36.388126853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:41:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:43.358046494 +0000 UTC m=+72.102180915" watchObservedRunningTime="2026-08-04 09:41:43.388126853 +0000 UTC m=+72.132261272" Aug 04 09:41:43.392666 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.392636 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95zwj\" (UniqueName: \"kubernetes.io/projected/7f40804f-b0dd-4a82-982c-da8591f7fdf5-kube-api-access-95zwj\") pod \"image-registry-588c6c96bb-gz8pg\" (UID: \"7f40804f-b0dd-4a82-982c-da8591f7fdf5\") " pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" Aug 04 09:41:43.392851 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.392835 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7f40804f-b0dd-4a82-982c-da8591f7fdf5-bound-sa-token\") pod \"image-registry-588c6c96bb-gz8pg\" (UID: \"7f40804f-b0dd-4a82-982c-da8591f7fdf5\") " pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" Aug 04 09:41:43.399106 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.399082 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:43.442825 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.442766 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bff96c9fd-q9gj4" Aug 04 09:41:43.513500 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.513472 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" Aug 04 09:41:43.536939 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.536901 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:43.588558 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.588279 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bff96c9fd-q9gj4"] Aug 04 09:41:43.590872 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:41:43.590843 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6fe622e3_2407_4db6_977f_6c8d1ece43f8.slice/crio-aa684f2c0ab567c964ced589afe8db06efd2a82f0ee0f6a36950689e3de486b7 WatchSource:0}: Error finding container aa684f2c0ab567c964ced589afe8db06efd2a82f0ee0f6a36950689e3de486b7: Status 404 returned error can't find the container with id aa684f2c0ab567c964ced589afe8db06efd2a82f0ee0f6a36950689e3de486b7 Aug 04 09:41:43.605133 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.605104 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:43.607743 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.607719 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:43.677988 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.677896 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-588c6c96bb-gz8pg"] Aug 04 09:41:43.683987 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:41:43.683953 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f40804f_b0dd_4a82_982c_da8591f7fdf5.slice/crio-5a649f846b62fb4a074339ab67d07b7c99ba7336260c9b17ee78f07aea2b16dd WatchSource:0}: Error finding container 5a649f846b62fb4a074339ab67d07b7c99ba7336260c9b17ee78f07aea2b16dd: Status 404 returned error can't find the container with id 5a649f846b62fb4a074339ab67d07b7c99ba7336260c9b17ee78f07aea2b16dd Aug 04 09:41:43.776480 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:43.776444 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:44.139120 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:44.139068 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bff96c9fd-q9gj4" event={"ID":"6fe622e3-2407-4db6-977f-6c8d1ece43f8","Type":"ContainerStarted","Data":"aa684f2c0ab567c964ced589afe8db06efd2a82f0ee0f6a36950689e3de486b7"} Aug 04 09:41:44.140616 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:44.140590 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" event={"ID":"7f40804f-b0dd-4a82-982c-da8591f7fdf5","Type":"ContainerStarted","Data":"dd84806602ec317493845a4017464c3ec76d21447df1a14cf75483380a797ad5"} Aug 04 09:41:44.140754 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:44.140664 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" event={"ID":"7f40804f-b0dd-4a82-982c-da8591f7fdf5","Type":"ContainerStarted","Data":"5a649f846b62fb4a074339ab67d07b7c99ba7336260c9b17ee78f07aea2b16dd"} Aug 04 09:41:44.141263 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:44.141244 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:44.142528 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:44.142508 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-69f7b748fb-vt9zm" Aug 04 09:41:44.169017 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:44.168969 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" podStartSLOduration=1.168955162 podStartE2EDuration="1.168955162s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:41:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:44.168051894 +0000 UTC m=+72.912186313" watchObservedRunningTime="2026-08-04 09:41:44.168955162 +0000 UTC m=+72.913089580" Aug 04 09:41:45.145485 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:45.145452 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" Aug 04 09:41:45.196054 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:45.196012 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nn7rk\" (UID: \"11e66e79-a3c2-4c7c-a283-0fc4ecffd574\") " pod="openshift-insights/insights-runtime-extractor-nn7rk" Aug 04 09:41:45.198905 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:45.198871 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11e66e79-a3c2-4c7c-a283-0fc4ecffd574-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nn7rk\" (UID: \"11e66e79-a3c2-4c7c-a283-0fc4ecffd574\") " pod="openshift-insights/insights-runtime-extractor-nn7rk" Aug 04 09:41:45.423427 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:45.423351 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-54lnd\"" Aug 04 09:41:45.430815 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:45.430776 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nn7rk" Aug 04 09:41:46.071821 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:46.071771 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nn7rk"] Aug 04 09:41:46.074992 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:41:46.074960 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11e66e79_a3c2_4c7c_a283_0fc4ecffd574.slice/crio-5e39afb3516f549ca264cc14cd1a2c32f20ca024fe91378f4a124d5e7784ff03 WatchSource:0}: Error finding container 5e39afb3516f549ca264cc14cd1a2c32f20ca024fe91378f4a124d5e7784ff03: Status 404 returned error can't find the container with id 5e39afb3516f549ca264cc14cd1a2c32f20ca024fe91378f4a124d5e7784ff03 Aug 04 09:41:46.149589 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:46.149561 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bff96c9fd-q9gj4" event={"ID":"6fe622e3-2407-4db6-977f-6c8d1ece43f8","Type":"ContainerStarted","Data":"f565dd9729d5006f1ead712186e263dd56816049a7743daf533fe7a179d835fe"} Aug 04 09:41:46.152100 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:46.152071 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nn7rk" event={"ID":"11e66e79-a3c2-4c7c-a283-0fc4ecffd574","Type":"ContainerStarted","Data":"0b745efc9b60eaf1a4f09d27acfd0a63ee5e7c54d139cdf063be5c6ff7aa3da7"} Aug 04 09:41:46.152214 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:46.152129 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nn7rk" event={"ID":"11e66e79-a3c2-4c7c-a283-0fc4ecffd574","Type":"ContainerStarted","Data":"5e39afb3516f549ca264cc14cd1a2c32f20ca024fe91378f4a124d5e7784ff03"} Aug 04 09:41:46.179951 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:46.179897 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bff96c9fd-q9gj4" podStartSLOduration=0.811654558 podStartE2EDuration="3.17988027s" totalImagesPullingTime="2.368225712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:41:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:46.179406607 +0000 UTC m=+74.923541026" watchObservedRunningTime="2026-08-04 09:41:46.17988027 +0000 UTC m=+74.924014689" Aug 04 09:41:46.664239 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:46.664203 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:46.767009 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:46.766976 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:47.158443 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:47.158402 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nn7rk" event={"ID":"11e66e79-a3c2-4c7c-a283-0fc4ecffd574","Type":"ContainerStarted","Data":"19cbfed1317e6febfc2a76a88e4fd161a023ecacff346435a7cddba32eed4dca"} Aug 04 09:41:48.164212 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:48.164123 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nn7rk" event={"ID":"11e66e79-a3c2-4c7c-a283-0fc4ecffd574","Type":"ContainerStarted","Data":"bccd6b2cf9032be41b5e223a1c87d4cfd9d1f707059e7c4ff2ec2523e2e66699"} Aug 04 09:41:48.314345 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:48.314310 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:48.389968 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:48.389916 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nn7rk" podStartSLOduration=33.687762845 podStartE2EDuration="35.389897214s" totalImagesPullingTime="1.702134369s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 09:41:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:48.199719488 +0000 UTC m=+76.943853924" watchObservedRunningTime="2026-08-04 09:41:48.389897214 +0000 UTC m=+77.134031633" Aug 04 09:41:48.390442 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:48.390428 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-zlccm"] Aug 04 09:41:48.392479 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:48.392460 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-758d7cb874-zlccm" Aug 04 09:41:48.400014 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:48.399991 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Aug 04 09:41:48.400134 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:48.400029 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Aug 04 09:41:48.400134 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:48.400072 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Aug 04 09:41:48.400134 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:48.400091 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Aug 04 09:41:48.400134 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:48.400077 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Aug 04 09:41:48.400395 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:48.400382 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-lk56r\"" Aug 04 09:41:48.423947 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:48.423866 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-zlccm"] Aug 04 09:41:48.425722 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:48.425694 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:48.521843 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:48.521777 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/190841b0-afa9-47c0-92ed-b4eb283ba6d5-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-zlccm\" (UID: \"190841b0-afa9-47c0-92ed-b4eb283ba6d5\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-zlccm" Aug 04 09:41:48.521843 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:48.521845 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/190841b0-afa9-47c0-92ed-b4eb283ba6d5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-zlccm\" (UID: \"190841b0-afa9-47c0-92ed-b4eb283ba6d5\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-zlccm" Aug 04 09:41:48.522078 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:48.521887 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bw27\" (UniqueName: \"kubernetes.io/projected/190841b0-afa9-47c0-92ed-b4eb283ba6d5-kube-api-access-2bw27\") pod \"prometheus-operator-758d7cb874-zlccm\" (UID: \"190841b0-afa9-47c0-92ed-b4eb283ba6d5\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-zlccm" Aug 04 09:41:48.522078 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:48.521960 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/190841b0-afa9-47c0-92ed-b4eb283ba6d5-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-zlccm\" (UID: \"190841b0-afa9-47c0-92ed-b4eb283ba6d5\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-zlccm" Aug 04 09:41:48.622932 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:48.622901 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/190841b0-afa9-47c0-92ed-b4eb283ba6d5-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-zlccm\" (UID: \"190841b0-afa9-47c0-92ed-b4eb283ba6d5\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-zlccm" Aug 04 09:41:48.623127 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:48.622943 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/190841b0-afa9-47c0-92ed-b4eb283ba6d5-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-zlccm\" (UID: \"190841b0-afa9-47c0-92ed-b4eb283ba6d5\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-zlccm" Aug 04 09:41:48.623127 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:48.622964 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/190841b0-afa9-47c0-92ed-b4eb283ba6d5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-zlccm\" (UID: \"190841b0-afa9-47c0-92ed-b4eb283ba6d5\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-zlccm" Aug 04 09:41:48.623127 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:48.622992 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bw27\" (UniqueName: \"kubernetes.io/projected/190841b0-afa9-47c0-92ed-b4eb283ba6d5-kube-api-access-2bw27\") pod \"prometheus-operator-758d7cb874-zlccm\" (UID: \"190841b0-afa9-47c0-92ed-b4eb283ba6d5\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-zlccm" Aug 04 09:41:48.623674 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:48.623645 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/190841b0-afa9-47c0-92ed-b4eb283ba6d5-metrics-client-ca\") pod \"prometheus-operator-758d7cb874-zlccm\" (UID: \"190841b0-afa9-47c0-92ed-b4eb283ba6d5\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-zlccm" Aug 04 09:41:48.625407 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:48.625384 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/190841b0-afa9-47c0-92ed-b4eb283ba6d5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-758d7cb874-zlccm\" (UID: \"190841b0-afa9-47c0-92ed-b4eb283ba6d5\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-zlccm" Aug 04 09:41:48.625498 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:48.625459 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/190841b0-afa9-47c0-92ed-b4eb283ba6d5-prometheus-operator-tls\") pod \"prometheus-operator-758d7cb874-zlccm\" (UID: \"190841b0-afa9-47c0-92ed-b4eb283ba6d5\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-zlccm" Aug 04 09:41:48.639592 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:48.639558 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bw27\" (UniqueName: \"kubernetes.io/projected/190841b0-afa9-47c0-92ed-b4eb283ba6d5-kube-api-access-2bw27\") pod \"prometheus-operator-758d7cb874-zlccm\" (UID: \"190841b0-afa9-47c0-92ed-b4eb283ba6d5\") " pod="openshift-monitoring/prometheus-operator-758d7cb874-zlccm" Aug 04 09:41:48.701564 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:48.701531 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-758d7cb874-zlccm" Aug 04 09:41:48.842348 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:48.842311 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-758d7cb874-zlccm"] Aug 04 09:41:48.845149 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:41:48.845118 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod190841b0_afa9_47c0_92ed_b4eb283ba6d5.slice/crio-97afbd6407dc8cc02dbf5bcc7df7cdf4c14aa40dd5b519a716276d13caf80523 WatchSource:0}: Error finding container 97afbd6407dc8cc02dbf5bcc7df7cdf4c14aa40dd5b519a716276d13caf80523: Status 404 returned error can't find the container with id 97afbd6407dc8cc02dbf5bcc7df7cdf4c14aa40dd5b519a716276d13caf80523 Aug 04 09:41:49.169507 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:49.169426 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-zlccm" event={"ID":"190841b0-afa9-47c0-92ed-b4eb283ba6d5","Type":"ContainerStarted","Data":"97afbd6407dc8cc02dbf5bcc7df7cdf4c14aa40dd5b519a716276d13caf80523"} Aug 04 09:41:50.174461 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:50.174427 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-zlccm" event={"ID":"190841b0-afa9-47c0-92ed-b4eb283ba6d5","Type":"ContainerStarted","Data":"7527391ccfd0e93a530fd33475d03859aee2cd7392c2d31a4f2a944f4bf9a25d"} Aug 04 09:41:50.490601 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:50.490566 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:50.650997 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:50.650962 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:51.179952 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:51.179915 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-758d7cb874-zlccm" event={"ID":"190841b0-afa9-47c0-92ed-b4eb283ba6d5","Type":"ContainerStarted","Data":"3a41b4a73124594e60aa7fc02fab82fc173ba3f3a7743979ce303f44d8c8112d"} Aug 04 09:41:51.221013 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:51.220962 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-758d7cb874-zlccm" podStartSLOduration=1.977638937 podStartE2EDuration="3.220941148s" totalImagesPullingTime="1.243302211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:41:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:51.220039269 +0000 UTC m=+79.964173700" watchObservedRunningTime="2026-08-04 09:41:51.220941148 +0000 UTC m=+79.965075567" Aug 04 09:41:52.259223 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:52.259188 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:52.612162 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:52.612075 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:52.766989 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:52.766949 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:53.016564 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.016523 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:53.085686 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.085647 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-zctwh"] Aug 04 09:41:53.090107 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.090075 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:53.094229 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.094202 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5gwbm\"" Aug 04 09:41:53.094465 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.094447 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Aug 04 09:41:53.094632 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.094606 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Aug 04 09:41:53.094824 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.094787 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Aug 04 09:41:53.139106 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.139070 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:53.143937 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.143903 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cmck8" Aug 04 09:41:53.155856 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.155826 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-metrics-client-ca\") pod \"node-exporter-zctwh\" (UID: \"c6322d16-7b64-41ee-8ff2-dbb858a7bc96\") " pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:53.156054 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.155870 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-root\") pod \"node-exporter-zctwh\" (UID: \"c6322d16-7b64-41ee-8ff2-dbb858a7bc96\") " pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:53.156054 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.155913 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zctwh\" (UID: \"c6322d16-7b64-41ee-8ff2-dbb858a7bc96\") " pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:53.156054 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.155961 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-node-exporter-accelerators-collector-config\") pod \"node-exporter-zctwh\" (UID: \"c6322d16-7b64-41ee-8ff2-dbb858a7bc96\") " pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:53.156054 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.155996 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-node-exporter-wtmp\") pod \"node-exporter-zctwh\" (UID: \"c6322d16-7b64-41ee-8ff2-dbb858a7bc96\") " pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:53.156054 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.156018 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxdzk\" (UniqueName: \"kubernetes.io/projected/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-kube-api-access-rxdzk\") pod \"node-exporter-zctwh\" (UID: \"c6322d16-7b64-41ee-8ff2-dbb858a7bc96\") " pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:53.156286 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.156058 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-node-exporter-textfile\") pod \"node-exporter-zctwh\" (UID: \"c6322d16-7b64-41ee-8ff2-dbb858a7bc96\") " pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:53.156286 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.156083 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-sys\") pod \"node-exporter-zctwh\" (UID: \"c6322d16-7b64-41ee-8ff2-dbb858a7bc96\") " pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:53.156286 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.156117 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-node-exporter-tls\") pod \"node-exporter-zctwh\" (UID: \"c6322d16-7b64-41ee-8ff2-dbb858a7bc96\") " pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:53.248943 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.248778 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:53.256844 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.256807 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zctwh\" (UID: \"c6322d16-7b64-41ee-8ff2-dbb858a7bc96\") " pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:53.257018 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.256864 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-node-exporter-accelerators-collector-config\") pod \"node-exporter-zctwh\" (UID: \"c6322d16-7b64-41ee-8ff2-dbb858a7bc96\") " pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:53.257018 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.256894 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-node-exporter-wtmp\") pod \"node-exporter-zctwh\" (UID: \"c6322d16-7b64-41ee-8ff2-dbb858a7bc96\") " pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:53.257018 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.256910 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxdzk\" (UniqueName: \"kubernetes.io/projected/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-kube-api-access-rxdzk\") pod \"node-exporter-zctwh\" (UID: \"c6322d16-7b64-41ee-8ff2-dbb858a7bc96\") " pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:53.257018 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.256948 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-node-exporter-textfile\") pod \"node-exporter-zctwh\" (UID: \"c6322d16-7b64-41ee-8ff2-dbb858a7bc96\") " pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:53.257018 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.256974 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-sys\") pod \"node-exporter-zctwh\" (UID: \"c6322d16-7b64-41ee-8ff2-dbb858a7bc96\") " pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:53.257018 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.257017 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-node-exporter-tls\") pod \"node-exporter-zctwh\" (UID: \"c6322d16-7b64-41ee-8ff2-dbb858a7bc96\") " pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:53.257331 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.257064 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-metrics-client-ca\") pod \"node-exporter-zctwh\" (UID: \"c6322d16-7b64-41ee-8ff2-dbb858a7bc96\") " pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:53.257331 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.257070 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-node-exporter-wtmp\") pod \"node-exporter-zctwh\" (UID: \"c6322d16-7b64-41ee-8ff2-dbb858a7bc96\") " pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:53.257331 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.257096 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-root\") pod \"node-exporter-zctwh\" (UID: \"c6322d16-7b64-41ee-8ff2-dbb858a7bc96\") " pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:53.257331 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.257164 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-root\") pod \"node-exporter-zctwh\" (UID: \"c6322d16-7b64-41ee-8ff2-dbb858a7bc96\") " pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:53.257331 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:53.257241 2572 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Aug 04 09:41:53.257331 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:41:53.257313 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-node-exporter-tls podName:c6322d16-7b64-41ee-8ff2-dbb858a7bc96 nodeName:}" failed. No retries permitted until 2026-08-04 09:41:53.757292396 +0000 UTC m=+82.501426808 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-node-exporter-tls") pod "node-exporter-zctwh" (UID: "c6322d16-7b64-41ee-8ff2-dbb858a7bc96") : secret "node-exporter-tls" not found Aug 04 09:41:53.257331 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.257324 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-node-exporter-textfile\") pod \"node-exporter-zctwh\" (UID: \"c6322d16-7b64-41ee-8ff2-dbb858a7bc96\") " pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:53.257669 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.257347 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-sys\") pod \"node-exporter-zctwh\" (UID: \"c6322d16-7b64-41ee-8ff2-dbb858a7bc96\") " pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:53.257951 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.257916 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-metrics-client-ca\") pod \"node-exporter-zctwh\" (UID: \"c6322d16-7b64-41ee-8ff2-dbb858a7bc96\") " pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:53.258120 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.258101 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-node-exporter-accelerators-collector-config\") pod \"node-exporter-zctwh\" (UID: \"c6322d16-7b64-41ee-8ff2-dbb858a7bc96\") " pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:53.259766 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.259747 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zctwh\" (UID: \"c6322d16-7b64-41ee-8ff2-dbb858a7bc96\") " pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:53.269311 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.269243 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxdzk\" (UniqueName: \"kubernetes.io/projected/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-kube-api-access-rxdzk\") pod \"node-exporter-zctwh\" (UID: \"c6322d16-7b64-41ee-8ff2-dbb858a7bc96\") " pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:53.413377 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.413344 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:53.520265 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.520185 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:53.716062 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.716007 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:53.761362 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.761331 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-node-exporter-tls\") pod \"node-exporter-zctwh\" (UID: \"c6322d16-7b64-41ee-8ff2-dbb858a7bc96\") " pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:53.763657 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.763631 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c6322d16-7b64-41ee-8ff2-dbb858a7bc96-node-exporter-tls\") pod \"node-exporter-zctwh\" (UID: \"c6322d16-7b64-41ee-8ff2-dbb858a7bc96\") " pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:53.834839 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.834736 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:53.935652 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.935614 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:53.947681 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.947649 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 04 09:41:53.953574 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.953549 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:53.956876 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.956852 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Aug 04 09:41:53.957392 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.957358 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Aug 04 09:41:53.957633 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.957568 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Aug 04 09:41:53.958050 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.957857 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Aug 04 09:41:53.958050 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.957960 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Aug 04 09:41:53.958050 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.957998 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Aug 04 09:41:53.958050 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.958022 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Aug 04 09:41:53.958330 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.958081 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7v7ch\"" Aug 04 09:41:53.958330 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.958013 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Aug 04 09:41:53.964696 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.958464 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Aug 04 09:41:53.969681 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:53.969652 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 04 09:41:54.002312 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.002276 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zctwh" Aug 04 09:41:54.014959 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:41:54.014930 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6322d16_7b64_41ee_8ff2_dbb858a7bc96.slice/crio-4dbc1631b27e976b9f9ceb3313e9685e45603fda00ad8e987fb9aae05368f929 WatchSource:0}: Error finding container 4dbc1631b27e976b9f9ceb3313e9685e45603fda00ad8e987fb9aae05368f929: Status 404 returned error can't find the container with id 4dbc1631b27e976b9f9ceb3313e9685e45603fda00ad8e987fb9aae05368f929 Aug 04 09:41:54.064413 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.064370 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-web-config\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.064413 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.064415 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.064626 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.064438 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.064626 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.064459 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-config-volume\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.064626 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.064507 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48rqz\" (UniqueName: \"kubernetes.io/projected/59fb9607-ffdc-4604-82eb-46b1187f6628-kube-api-access-48rqz\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.064626 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.064561 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/59fb9607-ffdc-4604-82eb-46b1187f6628-tls-assets\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.064626 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.064582 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59fb9607-ffdc-4604-82eb-46b1187f6628-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.064626 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.064610 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/59fb9607-ffdc-4604-82eb-46b1187f6628-config-out\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.064905 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.064636 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.064905 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.064669 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/59fb9607-ffdc-4604-82eb-46b1187f6628-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.064905 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.064692 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59fb9607-ffdc-4604-82eb-46b1187f6628-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.064905 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.064719 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.064905 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.064781 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.135365 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.135272 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:54.165954 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.165912 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/59fb9607-ffdc-4604-82eb-46b1187f6628-config-out\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.165954 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.165954 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.166172 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.166076 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/59fb9607-ffdc-4604-82eb-46b1187f6628-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.166172 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.166131 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59fb9607-ffdc-4604-82eb-46b1187f6628-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.166172 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.166163 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.166312 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.166193 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.166312 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.166271 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-web-config\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.166312 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.166300 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.166473 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.166336 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.166473 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.166366 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-config-volume\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.166473 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.166389 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48rqz\" (UniqueName: \"kubernetes.io/projected/59fb9607-ffdc-4604-82eb-46b1187f6628-kube-api-access-48rqz\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.166473 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.166414 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/59fb9607-ffdc-4604-82eb-46b1187f6628-tls-assets\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.166473 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.166436 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59fb9607-ffdc-4604-82eb-46b1187f6628-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.166945 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.166916 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/59fb9607-ffdc-4604-82eb-46b1187f6628-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.167325 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.167297 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59fb9607-ffdc-4604-82eb-46b1187f6628-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.168123 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.168098 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59fb9607-ffdc-4604-82eb-46b1187f6628-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.169213 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.169185 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/59fb9607-ffdc-4604-82eb-46b1187f6628-config-out\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.169525 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.169473 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.169614 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.169572 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.169721 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.169695 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.170395 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.170370 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-web-config\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.170459 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.170442 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/59fb9607-ffdc-4604-82eb-46b1187f6628-tls-assets\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.170528 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.170509 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.170577 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.170544 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.170876 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.170859 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-config-volume\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.174928 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.174907 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48rqz\" (UniqueName: \"kubernetes.io/projected/59fb9607-ffdc-4604-82eb-46b1187f6628-kube-api-access-48rqz\") pod \"alertmanager-main-0\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.191857 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.191825 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zctwh" event={"ID":"c6322d16-7b64-41ee-8ff2-dbb858a7bc96","Type":"ContainerStarted","Data":"4dbc1631b27e976b9f9ceb3313e9685e45603fda00ad8e987fb9aae05368f929"} Aug 04 09:41:54.266040 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.265999 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:41:54.430699 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:54.430627 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 04 09:41:54.432730 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:41:54.432697 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59fb9607_ffdc_4604_82eb_46b1187f6628.slice/crio-d8b14bb90244e388630f212c2714535c75ca0198fdff1302f433d168b606928e WatchSource:0}: Error finding container d8b14bb90244e388630f212c2714535c75ca0198fdff1302f433d168b606928e: Status 404 returned error can't find the container with id d8b14bb90244e388630f212c2714535c75ca0198fdff1302f433d168b606928e Aug 04 09:41:55.197589 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:55.197560 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"59fb9607-ffdc-4604-82eb-46b1187f6628","Type":"ContainerStarted","Data":"d8b14bb90244e388630f212c2714535c75ca0198fdff1302f433d168b606928e"} Aug 04 09:41:55.858315 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:55.858281 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:55.959597 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:55.959566 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:56.077781 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:56.077741 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:56.172280 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:56.172179 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:56.202896 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:56.202862 2572 generic.go:356] "Generic (PLEG): container finished" podID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerID="48be1d4a8fd45aba311286adc7583d434189c35f4e769e31fd109ee1b9f0634f" exitCode=0 Aug 04 09:41:56.203062 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:56.202925 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"59fb9607-ffdc-4604-82eb-46b1187f6628","Type":"ContainerDied","Data":"48be1d4a8fd45aba311286adc7583d434189c35f4e769e31fd109ee1b9f0634f"} Aug 04 09:41:56.204912 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:56.204887 2572 generic.go:356] "Generic (PLEG): container finished" podID="c6322d16-7b64-41ee-8ff2-dbb858a7bc96" containerID="0193730f9b1c5fd6d0d4584d4baf3bda6dc6c8934dfdc463ae36a1b59204dc8c" exitCode=0 Aug 04 09:41:56.204991 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:56.204951 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zctwh" event={"ID":"c6322d16-7b64-41ee-8ff2-dbb858a7bc96","Type":"ContainerDied","Data":"0193730f9b1c5fd6d0d4584d4baf3bda6dc6c8934dfdc463ae36a1b59204dc8c"} Aug 04 09:41:56.290300 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:56.290266 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:56.599150 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:56.599114 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:57.211588 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.211549 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zctwh" event={"ID":"c6322d16-7b64-41ee-8ff2-dbb858a7bc96","Type":"ContainerStarted","Data":"ae51948c996bd101c4186bc7a6fbea49a25e4701c7b074f27fe7bdb9010a8a1d"} Aug 04 09:41:57.212033 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.211649 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zctwh" event={"ID":"c6322d16-7b64-41ee-8ff2-dbb858a7bc96","Type":"ContainerStarted","Data":"1965abff0050ab05c0f270a9f5f1bf67ef3b70d0e9cb0f8c516203e15e5bb601"} Aug 04 09:41:57.237708 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.237652 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-zctwh" podStartSLOduration=3.042112365 podStartE2EDuration="4.237634875s" totalImagesPullingTime="1.120395844s" totalInitContainerRuntime="75.126666ms" isStatefulPod=true podCreationTimestamp="2026-08-04 09:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:57.235667501 +0000 UTC m=+85.979801944" watchObservedRunningTime="2026-08-04 09:41:57.237634875 +0000 UTC m=+85.981769298" Aug 04 09:41:57.581949 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.581866 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:57.607245 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.607210 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-757d6d8489-6kbhk"] Aug 04 09:41:57.611858 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.611825 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" Aug 04 09:41:57.622108 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.622079 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Aug 04 09:41:57.625067 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.623450 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Aug 04 09:41:57.625067 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.623530 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-e3om6j0aidp3j\"" Aug 04 09:41:57.625067 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.623937 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Aug 04 09:41:57.625067 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.625019 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Aug 04 09:41:57.625345 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.625279 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-vvtvz\"" Aug 04 09:41:57.633031 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.633005 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-757d6d8489-6kbhk"] Aug 04 09:41:57.697678 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.697642 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/102ce85d-672a-4526-bdae-be62d5d43c4c-audit-log\") pod \"metrics-server-757d6d8489-6kbhk\" (UID: \"102ce85d-672a-4526-bdae-be62d5d43c4c\") " pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" Aug 04 09:41:57.697678 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.697683 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/102ce85d-672a-4526-bdae-be62d5d43c4c-client-ca-bundle\") pod \"metrics-server-757d6d8489-6kbhk\" (UID: \"102ce85d-672a-4526-bdae-be62d5d43c4c\") " pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" Aug 04 09:41:57.697932 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.697786 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/102ce85d-672a-4526-bdae-be62d5d43c4c-secret-metrics-server-client-certs\") pod \"metrics-server-757d6d8489-6kbhk\" (UID: \"102ce85d-672a-4526-bdae-be62d5d43c4c\") " pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" Aug 04 09:41:57.697932 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.697850 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/102ce85d-672a-4526-bdae-be62d5d43c4c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-757d6d8489-6kbhk\" (UID: \"102ce85d-672a-4526-bdae-be62d5d43c4c\") " pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" Aug 04 09:41:57.697932 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.697914 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/102ce85d-672a-4526-bdae-be62d5d43c4c-secret-metrics-server-tls\") pod \"metrics-server-757d6d8489-6kbhk\" (UID: \"102ce85d-672a-4526-bdae-be62d5d43c4c\") " pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" Aug 04 09:41:57.698103 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.697946 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/102ce85d-672a-4526-bdae-be62d5d43c4c-metrics-server-audit-profiles\") pod \"metrics-server-757d6d8489-6kbhk\" (UID: \"102ce85d-672a-4526-bdae-be62d5d43c4c\") " pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" Aug 04 09:41:57.698103 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.697983 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88tlj\" (UniqueName: \"kubernetes.io/projected/102ce85d-672a-4526-bdae-be62d5d43c4c-kube-api-access-88tlj\") pod \"metrics-server-757d6d8489-6kbhk\" (UID: \"102ce85d-672a-4526-bdae-be62d5d43c4c\") " pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" Aug 04 09:41:57.799109 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.799081 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/102ce85d-672a-4526-bdae-be62d5d43c4c-secret-metrics-server-client-certs\") pod \"metrics-server-757d6d8489-6kbhk\" (UID: \"102ce85d-672a-4526-bdae-be62d5d43c4c\") " pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" Aug 04 09:41:57.799267 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.799115 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/102ce85d-672a-4526-bdae-be62d5d43c4c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-757d6d8489-6kbhk\" (UID: \"102ce85d-672a-4526-bdae-be62d5d43c4c\") " pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" Aug 04 09:41:57.799267 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.799168 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/102ce85d-672a-4526-bdae-be62d5d43c4c-secret-metrics-server-tls\") pod \"metrics-server-757d6d8489-6kbhk\" (UID: \"102ce85d-672a-4526-bdae-be62d5d43c4c\") " pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" Aug 04 09:41:57.799267 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.799197 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/102ce85d-672a-4526-bdae-be62d5d43c4c-metrics-server-audit-profiles\") pod \"metrics-server-757d6d8489-6kbhk\" (UID: \"102ce85d-672a-4526-bdae-be62d5d43c4c\") " pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" Aug 04 09:41:57.799267 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.799229 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88tlj\" (UniqueName: \"kubernetes.io/projected/102ce85d-672a-4526-bdae-be62d5d43c4c-kube-api-access-88tlj\") pod \"metrics-server-757d6d8489-6kbhk\" (UID: \"102ce85d-672a-4526-bdae-be62d5d43c4c\") " pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" Aug 04 09:41:57.799463 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.799282 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/102ce85d-672a-4526-bdae-be62d5d43c4c-audit-log\") pod \"metrics-server-757d6d8489-6kbhk\" (UID: \"102ce85d-672a-4526-bdae-be62d5d43c4c\") " pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" Aug 04 09:41:57.799463 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.799306 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/102ce85d-672a-4526-bdae-be62d5d43c4c-client-ca-bundle\") pod \"metrics-server-757d6d8489-6kbhk\" (UID: \"102ce85d-672a-4526-bdae-be62d5d43c4c\") " pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" Aug 04 09:41:57.799960 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.799928 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/102ce85d-672a-4526-bdae-be62d5d43c4c-audit-log\") pod \"metrics-server-757d6d8489-6kbhk\" (UID: \"102ce85d-672a-4526-bdae-be62d5d43c4c\") " pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" Aug 04 09:41:57.800523 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.800284 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/102ce85d-672a-4526-bdae-be62d5d43c4c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-757d6d8489-6kbhk\" (UID: \"102ce85d-672a-4526-bdae-be62d5d43c4c\") " pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" Aug 04 09:41:57.800633 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.800551 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/102ce85d-672a-4526-bdae-be62d5d43c4c-metrics-server-audit-profiles\") pod \"metrics-server-757d6d8489-6kbhk\" (UID: \"102ce85d-672a-4526-bdae-be62d5d43c4c\") " pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" Aug 04 09:41:57.801631 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.801609 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/102ce85d-672a-4526-bdae-be62d5d43c4c-secret-metrics-server-client-certs\") pod \"metrics-server-757d6d8489-6kbhk\" (UID: \"102ce85d-672a-4526-bdae-be62d5d43c4c\") " pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" Aug 04 09:41:57.802094 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.802073 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/102ce85d-672a-4526-bdae-be62d5d43c4c-client-ca-bundle\") pod \"metrics-server-757d6d8489-6kbhk\" (UID: \"102ce85d-672a-4526-bdae-be62d5d43c4c\") " pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" Aug 04 09:41:57.802198 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.802070 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/102ce85d-672a-4526-bdae-be62d5d43c4c-secret-metrics-server-tls\") pod \"metrics-server-757d6d8489-6kbhk\" (UID: \"102ce85d-672a-4526-bdae-be62d5d43c4c\") " pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" Aug 04 09:41:57.807824 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.807784 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88tlj\" (UniqueName: \"kubernetes.io/projected/102ce85d-672a-4526-bdae-be62d5d43c4c-kube-api-access-88tlj\") pod \"metrics-server-757d6d8489-6kbhk\" (UID: \"102ce85d-672a-4526-bdae-be62d5d43c4c\") " pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" Aug 04 09:41:57.922952 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:57.922914 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" Aug 04 09:41:58.086487 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.086454 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-757d6d8489-6kbhk"] Aug 04 09:41:58.089279 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:41:58.089244 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod102ce85d_672a_4526_bdae_be62d5d43c4c.slice/crio-98cfa7bae374518582ebb8772d3b50a087720a1d225b219ce9e24a54d9e13960 WatchSource:0}: Error finding container 98cfa7bae374518582ebb8772d3b50a087720a1d225b219ce9e24a54d9e13960: Status 404 returned error can't find the container with id 98cfa7bae374518582ebb8772d3b50a087720a1d225b219ce9e24a54d9e13960 Aug 04 09:41:58.193331 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.193294 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:58.217626 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.217593 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" event={"ID":"102ce85d-672a-4526-bdae-be62d5d43c4c","Type":"ContainerStarted","Data":"98cfa7bae374518582ebb8772d3b50a087720a1d225b219ce9e24a54d9e13960"} Aug 04 09:41:58.220733 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.220701 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"59fb9607-ffdc-4604-82eb-46b1187f6628","Type":"ContainerStarted","Data":"933639ee0fcafa4406f522e098f1a38b24dda4420d63c49c8dd5be616cef0fa2"} Aug 04 09:41:58.220889 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.220768 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"59fb9607-ffdc-4604-82eb-46b1187f6628","Type":"ContainerStarted","Data":"cd14f36db6dd8646424ebcd7cd6e8f2de489f47d7615089be7769a44440ce18b"} Aug 04 09:41:58.220889 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.220807 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"59fb9607-ffdc-4604-82eb-46b1187f6628","Type":"ContainerStarted","Data":"8b70ccb8a6557d65e7319700e015a35ee0c86387beabeb4aa491b7e8accfce70"} Aug 04 09:41:58.220889 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.220843 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"59fb9607-ffdc-4604-82eb-46b1187f6628","Type":"ContainerStarted","Data":"2058ff56324d4814eba7505df61abbd33ef778a1c4853349732031ad9ba937f1"} Aug 04 09:41:58.220889 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.220874 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"59fb9607-ffdc-4604-82eb-46b1187f6628","Type":"ContainerStarted","Data":"8f23cd1f0e7b8e087b3de416f2284535a5b1e5236a0430b8f8982610a6066cab"} Aug 04 09:41:58.249232 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.249192 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7c8459789c-glxzj"] Aug 04 09:41:58.254442 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.254064 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" Aug 04 09:41:58.257196 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.257161 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-k8qmb\"" Aug 04 09:41:58.257196 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.257169 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Aug 04 09:41:58.257396 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.257209 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Aug 04 09:41:58.257396 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.257229 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Aug 04 09:41:58.257396 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.257241 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Aug 04 09:41:58.257396 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.257302 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Aug 04 09:41:58.262146 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.261997 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Aug 04 09:41:58.271553 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.271525 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7c8459789c-glxzj"] Aug 04 09:41:58.404016 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.403921 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d404891b-cd94-4e3d-8955-3734c1265f34-secret-telemeter-client\") pod \"telemeter-client-7c8459789c-glxzj\" (UID: \"d404891b-cd94-4e3d-8955-3734c1265f34\") " pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" Aug 04 09:41:58.404016 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.403964 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d404891b-cd94-4e3d-8955-3734c1265f34-serving-certs-ca-bundle\") pod \"telemeter-client-7c8459789c-glxzj\" (UID: \"d404891b-cd94-4e3d-8955-3734c1265f34\") " pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" Aug 04 09:41:58.404226 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.404086 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d404891b-cd94-4e3d-8955-3734c1265f34-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c8459789c-glxzj\" (UID: \"d404891b-cd94-4e3d-8955-3734c1265f34\") " pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" Aug 04 09:41:58.404226 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.404127 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d404891b-cd94-4e3d-8955-3734c1265f34-telemeter-client-tls\") pod \"telemeter-client-7c8459789c-glxzj\" (UID: \"d404891b-cd94-4e3d-8955-3734c1265f34\") " pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" Aug 04 09:41:58.404226 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.404147 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d404891b-cd94-4e3d-8955-3734c1265f34-federate-client-tls\") pod \"telemeter-client-7c8459789c-glxzj\" (UID: \"d404891b-cd94-4e3d-8955-3734c1265f34\") " pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" Aug 04 09:41:58.404226 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.404212 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcxjq\" (UniqueName: \"kubernetes.io/projected/d404891b-cd94-4e3d-8955-3734c1265f34-kube-api-access-mcxjq\") pod \"telemeter-client-7c8459789c-glxzj\" (UID: \"d404891b-cd94-4e3d-8955-3734c1265f34\") " pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" Aug 04 09:41:58.404377 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.404252 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d404891b-cd94-4e3d-8955-3734c1265f34-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c8459789c-glxzj\" (UID: \"d404891b-cd94-4e3d-8955-3734c1265f34\") " pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" Aug 04 09:41:58.404377 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.404278 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d404891b-cd94-4e3d-8955-3734c1265f34-metrics-client-ca\") pod \"telemeter-client-7c8459789c-glxzj\" (UID: \"d404891b-cd94-4e3d-8955-3734c1265f34\") " pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" Aug 04 09:41:58.505226 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.505180 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d404891b-cd94-4e3d-8955-3734c1265f34-telemeter-client-tls\") pod \"telemeter-client-7c8459789c-glxzj\" (UID: \"d404891b-cd94-4e3d-8955-3734c1265f34\") " pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" Aug 04 09:41:58.505226 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.505231 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d404891b-cd94-4e3d-8955-3734c1265f34-federate-client-tls\") pod \"telemeter-client-7c8459789c-glxzj\" (UID: \"d404891b-cd94-4e3d-8955-3734c1265f34\") " pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" Aug 04 09:41:58.505590 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.505280 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcxjq\" (UniqueName: \"kubernetes.io/projected/d404891b-cd94-4e3d-8955-3734c1265f34-kube-api-access-mcxjq\") pod \"telemeter-client-7c8459789c-glxzj\" (UID: \"d404891b-cd94-4e3d-8955-3734c1265f34\") " pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" Aug 04 09:41:58.505590 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.505307 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d404891b-cd94-4e3d-8955-3734c1265f34-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c8459789c-glxzj\" (UID: \"d404891b-cd94-4e3d-8955-3734c1265f34\") " pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" Aug 04 09:41:58.505590 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.505337 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d404891b-cd94-4e3d-8955-3734c1265f34-metrics-client-ca\") pod \"telemeter-client-7c8459789c-glxzj\" (UID: \"d404891b-cd94-4e3d-8955-3734c1265f34\") " pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" Aug 04 09:41:58.505590 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.505370 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d404891b-cd94-4e3d-8955-3734c1265f34-secret-telemeter-client\") pod \"telemeter-client-7c8459789c-glxzj\" (UID: \"d404891b-cd94-4e3d-8955-3734c1265f34\") " pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" Aug 04 09:41:58.505590 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.505395 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d404891b-cd94-4e3d-8955-3734c1265f34-serving-certs-ca-bundle\") pod \"telemeter-client-7c8459789c-glxzj\" (UID: \"d404891b-cd94-4e3d-8955-3734c1265f34\") " pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" Aug 04 09:41:58.505590 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.505506 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d404891b-cd94-4e3d-8955-3734c1265f34-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c8459789c-glxzj\" (UID: \"d404891b-cd94-4e3d-8955-3734c1265f34\") " pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" Aug 04 09:41:58.506318 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.506287 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d404891b-cd94-4e3d-8955-3734c1265f34-serving-certs-ca-bundle\") pod \"telemeter-client-7c8459789c-glxzj\" (UID: \"d404891b-cd94-4e3d-8955-3734c1265f34\") " pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" Aug 04 09:41:58.506440 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.506287 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d404891b-cd94-4e3d-8955-3734c1265f34-metrics-client-ca\") pod \"telemeter-client-7c8459789c-glxzj\" (UID: \"d404891b-cd94-4e3d-8955-3734c1265f34\") " pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" Aug 04 09:41:58.506440 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.506327 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d404891b-cd94-4e3d-8955-3734c1265f34-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c8459789c-glxzj\" (UID: \"d404891b-cd94-4e3d-8955-3734c1265f34\") " pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" Aug 04 09:41:58.507927 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.507895 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d404891b-cd94-4e3d-8955-3734c1265f34-federate-client-tls\") pod \"telemeter-client-7c8459789c-glxzj\" (UID: \"d404891b-cd94-4e3d-8955-3734c1265f34\") " pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" Aug 04 09:41:58.508036 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.507966 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d404891b-cd94-4e3d-8955-3734c1265f34-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c8459789c-glxzj\" (UID: \"d404891b-cd94-4e3d-8955-3734c1265f34\") " pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" Aug 04 09:41:58.508223 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.508205 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d404891b-cd94-4e3d-8955-3734c1265f34-telemeter-client-tls\") pod \"telemeter-client-7c8459789c-glxzj\" (UID: \"d404891b-cd94-4e3d-8955-3734c1265f34\") " pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" Aug 04 09:41:58.508338 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.508318 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d404891b-cd94-4e3d-8955-3734c1265f34-secret-telemeter-client\") pod \"telemeter-client-7c8459789c-glxzj\" (UID: \"d404891b-cd94-4e3d-8955-3734c1265f34\") " pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" Aug 04 09:41:58.518415 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.518391 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcxjq\" (UniqueName: \"kubernetes.io/projected/d404891b-cd94-4e3d-8955-3734c1265f34-kube-api-access-mcxjq\") pod \"telemeter-client-7c8459789c-glxzj\" (UID: \"d404891b-cd94-4e3d-8955-3734c1265f34\") " pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" Aug 04 09:41:58.564614 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.564581 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" Aug 04 09:41:58.718612 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.718441 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7c8459789c-glxzj"] Aug 04 09:41:58.721599 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:41:58.721560 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd404891b_cd94_4e3d_8955_3734c1265f34.slice/crio-6f0a0c8130695c00e981021965a0e7c16c2ead9985f0bca4be20fbabba380b45 WatchSource:0}: Error finding container 6f0a0c8130695c00e981021965a0e7c16c2ead9985f0bca4be20fbabba380b45: Status 404 returned error can't find the container with id 6f0a0c8130695c00e981021965a0e7c16c2ead9985f0bca4be20fbabba380b45 Aug 04 09:41:58.920640 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:58.920602 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:59.011441 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:59.011409 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:59.225732 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:59.225624 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" event={"ID":"d404891b-cd94-4e3d-8955-3734c1265f34","Type":"ContainerStarted","Data":"6f0a0c8130695c00e981021965a0e7c16c2ead9985f0bca4be20fbabba380b45"} Aug 04 09:41:59.229969 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:59.229934 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"59fb9607-ffdc-4604-82eb-46b1187f6628","Type":"ContainerStarted","Data":"1839c183a96aebc21fc547594fefe37f11ce0cecef1877f6396ae47c14a7187a"} Aug 04 09:41:59.274608 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:59.274543 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.890829878 podStartE2EDuration="6.274519518s" totalImagesPullingTime="3.367884372s" totalInitContainerRuntime="15.805268ms" isStatefulPod=false podCreationTimestamp="2026-08-04 09:41:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:41:59.272538504 +0000 UTC m=+88.016672926" watchObservedRunningTime="2026-08-04 09:41:59.274519518 +0000 UTC m=+88.018653939" Aug 04 09:41:59.438246 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:59.438176 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:59.709841 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:59.709771 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:41:59.797024 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:41:59.796973 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:00.236870 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:00.236758 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" event={"ID":"102ce85d-672a-4526-bdae-be62d5d43c4c","Type":"ContainerStarted","Data":"9605a9f381b1dd3ff2072f1a4bf4845ea733115874ac748c304751fb8471ddfb"} Aug 04 09:42:00.259163 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:00.259106 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" podStartSLOduration=1.412043947 podStartE2EDuration="3.259087064s" totalImagesPullingTime="1.847043117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:41:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:42:00.258190772 +0000 UTC m=+89.002325200" watchObservedRunningTime="2026-08-04 09:42:00.259087064 +0000 UTC m=+89.003221534" Aug 04 09:42:01.214876 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:01.214835 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-77c974ffcf-bg556"] Aug 04 09:42:01.218178 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:01.218156 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-bg556" Aug 04 09:42:01.225221 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:01.225201 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Aug 04 09:42:01.225310 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:01.225295 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Aug 04 09:42:01.225420 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:01.225403 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-7p57v\"" Aug 04 09:42:01.243774 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:01.243594 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-bg556"] Aug 04 09:42:01.247180 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:01.247143 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" event={"ID":"d404891b-cd94-4e3d-8955-3734c1265f34","Type":"ContainerStarted","Data":"70f1441b92a366a2f3eaae2a7e374d7bb347857e2cbc9aefa4069d626ac12166"} Aug 04 09:42:01.247320 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:01.247208 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" event={"ID":"d404891b-cd94-4e3d-8955-3734c1265f34","Type":"ContainerStarted","Data":"d9b1c28b9655f056019424d063d84b317b6b1e8f3f61bc7436d4e936d20f1cc7"} Aug 04 09:42:01.247320 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:01.247228 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" event={"ID":"d404891b-cd94-4e3d-8955-3734c1265f34","Type":"ContainerStarted","Data":"4770cceb045411f28301ed3d30c31d1db72bac356b94b71c8f4b1df45698e573"} Aug 04 09:42:01.255581 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:01.255545 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:01.336857 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:01.336819 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kg4p\" (UniqueName: \"kubernetes.io/projected/8304a19e-b591-4a42-aecb-13e8bc84f5b9-kube-api-access-7kg4p\") pod \"downloads-77c974ffcf-bg556\" (UID: \"8304a19e-b591-4a42-aecb-13e8bc84f5b9\") " pod="openshift-console/downloads-77c974ffcf-bg556" Aug 04 09:42:01.415002 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:01.414964 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:01.438514 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:01.438481 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kg4p\" (UniqueName: \"kubernetes.io/projected/8304a19e-b591-4a42-aecb-13e8bc84f5b9-kube-api-access-7kg4p\") pod \"downloads-77c974ffcf-bg556\" (UID: \"8304a19e-b591-4a42-aecb-13e8bc84f5b9\") " pod="openshift-console/downloads-77c974ffcf-bg556" Aug 04 09:42:01.455255 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:01.455219 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kg4p\" (UniqueName: \"kubernetes.io/projected/8304a19e-b591-4a42-aecb-13e8bc84f5b9-kube-api-access-7kg4p\") pod \"downloads-77c974ffcf-bg556\" (UID: \"8304a19e-b591-4a42-aecb-13e8bc84f5b9\") " pod="openshift-console/downloads-77c974ffcf-bg556" Aug 04 09:42:01.527201 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:01.527110 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-77c974ffcf-bg556" Aug 04 09:42:01.529313 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:01.529284 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:01.663096 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:01.663047 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7c8459789c-glxzj" podStartSLOduration=1.7150014740000001 podStartE2EDuration="3.663027647s" totalImagesPullingTime="1.948026173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:41:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:42:01.298154826 +0000 UTC m=+90.042289245" watchObservedRunningTime="2026-08-04 09:42:01.663027647 +0000 UTC m=+90.407162066" Aug 04 09:42:01.663844 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:01.663821 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-77c974ffcf-bg556"] Aug 04 09:42:01.665618 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:42:01.665593 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8304a19e_b591_4a42_aecb_13e8bc84f5b9.slice/crio-fc9ddb85a9412f26d8761c6112e5136517b213a5e512dfd6de36f860ae1821d1 WatchSource:0}: Error finding container fc9ddb85a9412f26d8761c6112e5136517b213a5e512dfd6de36f860ae1821d1: Status 404 returned error can't find the container with id fc9ddb85a9412f26d8761c6112e5136517b213a5e512dfd6de36f860ae1821d1 Aug 04 09:42:01.672332 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:01.672299 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:01.767001 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:01.766971 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:02.100024 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:02.099993 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:02.202912 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:02.202877 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:02.259558 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:02.259484 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-bg556" event={"ID":"8304a19e-b591-4a42-aecb-13e8bc84f5b9","Type":"ContainerStarted","Data":"fc9ddb85a9412f26d8761c6112e5136517b213a5e512dfd6de36f860ae1821d1"} Aug 04 09:42:02.339527 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:02.339484 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:02.947487 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:02.947445 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:03.174243 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:03.174140 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:03.343465 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:03.343383 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:03.459138 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:03.459100 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:06.157177 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:06.157138 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-588c6c96bb-gz8pg" Aug 04 09:42:06.525550 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:06.525515 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:06.622007 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:06.621963 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:07.007279 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:07.007238 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f7fcd76f7-q7spb"] Aug 04 09:42:07.010942 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:07.010914 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f7fcd76f7-q7spb" Aug 04 09:42:07.014752 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:07.014716 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Aug 04 09:42:07.016325 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:07.016275 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-8x9rc\"" Aug 04 09:42:07.016922 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:07.016654 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Aug 04 09:42:07.016922 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:07.016688 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Aug 04 09:42:07.017106 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:07.017015 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Aug 04 09:42:07.017160 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:07.017105 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Aug 04 09:42:07.023502 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:07.023471 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f7fcd76f7-q7spb"] Aug 04 09:42:07.195487 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:07.195443 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62ac4e5c-8f17-400e-bd44-026f3c769e90-console-serving-cert\") pod \"console-7f7fcd76f7-q7spb\" (UID: \"62ac4e5c-8f17-400e-bd44-026f3c769e90\") " pod="openshift-console/console-7f7fcd76f7-q7spb" Aug 04 09:42:07.195999 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:07.195518 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t27z\" (UniqueName: \"kubernetes.io/projected/62ac4e5c-8f17-400e-bd44-026f3c769e90-kube-api-access-9t27z\") pod \"console-7f7fcd76f7-q7spb\" (UID: \"62ac4e5c-8f17-400e-bd44-026f3c769e90\") " pod="openshift-console/console-7f7fcd76f7-q7spb" Aug 04 09:42:07.195999 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:07.195552 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62ac4e5c-8f17-400e-bd44-026f3c769e90-oauth-serving-cert\") pod \"console-7f7fcd76f7-q7spb\" (UID: \"62ac4e5c-8f17-400e-bd44-026f3c769e90\") " pod="openshift-console/console-7f7fcd76f7-q7spb" Aug 04 09:42:07.195999 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:07.195686 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62ac4e5c-8f17-400e-bd44-026f3c769e90-service-ca\") pod \"console-7f7fcd76f7-q7spb\" (UID: \"62ac4e5c-8f17-400e-bd44-026f3c769e90\") " pod="openshift-console/console-7f7fcd76f7-q7spb" Aug 04 09:42:07.195999 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:07.195737 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62ac4e5c-8f17-400e-bd44-026f3c769e90-console-config\") pod \"console-7f7fcd76f7-q7spb\" (UID: \"62ac4e5c-8f17-400e-bd44-026f3c769e90\") " pod="openshift-console/console-7f7fcd76f7-q7spb" Aug 04 09:42:07.195999 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:07.195862 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62ac4e5c-8f17-400e-bd44-026f3c769e90-console-oauth-config\") pod \"console-7f7fcd76f7-q7spb\" (UID: \"62ac4e5c-8f17-400e-bd44-026f3c769e90\") " pod="openshift-console/console-7f7fcd76f7-q7spb" Aug 04 09:42:07.297202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:07.297112 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t27z\" (UniqueName: \"kubernetes.io/projected/62ac4e5c-8f17-400e-bd44-026f3c769e90-kube-api-access-9t27z\") pod \"console-7f7fcd76f7-q7spb\" (UID: \"62ac4e5c-8f17-400e-bd44-026f3c769e90\") " pod="openshift-console/console-7f7fcd76f7-q7spb" Aug 04 09:42:07.297202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:07.297167 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62ac4e5c-8f17-400e-bd44-026f3c769e90-oauth-serving-cert\") pod \"console-7f7fcd76f7-q7spb\" (UID: \"62ac4e5c-8f17-400e-bd44-026f3c769e90\") " pod="openshift-console/console-7f7fcd76f7-q7spb" Aug 04 09:42:07.297202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:07.297199 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62ac4e5c-8f17-400e-bd44-026f3c769e90-service-ca\") pod \"console-7f7fcd76f7-q7spb\" (UID: \"62ac4e5c-8f17-400e-bd44-026f3c769e90\") " pod="openshift-console/console-7f7fcd76f7-q7spb" Aug 04 09:42:07.297481 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:07.297220 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62ac4e5c-8f17-400e-bd44-026f3c769e90-console-config\") pod \"console-7f7fcd76f7-q7spb\" (UID: \"62ac4e5c-8f17-400e-bd44-026f3c769e90\") " pod="openshift-console/console-7f7fcd76f7-q7spb" Aug 04 09:42:07.297481 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:07.297254 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62ac4e5c-8f17-400e-bd44-026f3c769e90-console-oauth-config\") pod \"console-7f7fcd76f7-q7spb\" (UID: \"62ac4e5c-8f17-400e-bd44-026f3c769e90\") " pod="openshift-console/console-7f7fcd76f7-q7spb" Aug 04 09:42:07.297481 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:07.297315 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62ac4e5c-8f17-400e-bd44-026f3c769e90-console-serving-cert\") pod \"console-7f7fcd76f7-q7spb\" (UID: \"62ac4e5c-8f17-400e-bd44-026f3c769e90\") " pod="openshift-console/console-7f7fcd76f7-q7spb" Aug 04 09:42:07.298147 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:07.298093 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62ac4e5c-8f17-400e-bd44-026f3c769e90-console-config\") pod \"console-7f7fcd76f7-q7spb\" (UID: \"62ac4e5c-8f17-400e-bd44-026f3c769e90\") " pod="openshift-console/console-7f7fcd76f7-q7spb" Aug 04 09:42:07.298147 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:07.298096 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62ac4e5c-8f17-400e-bd44-026f3c769e90-oauth-serving-cert\") pod \"console-7f7fcd76f7-q7spb\" (UID: \"62ac4e5c-8f17-400e-bd44-026f3c769e90\") " pod="openshift-console/console-7f7fcd76f7-q7spb" Aug 04 09:42:07.298323 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:07.298161 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62ac4e5c-8f17-400e-bd44-026f3c769e90-service-ca\") pod \"console-7f7fcd76f7-q7spb\" (UID: \"62ac4e5c-8f17-400e-bd44-026f3c769e90\") " pod="openshift-console/console-7f7fcd76f7-q7spb" Aug 04 09:42:07.300146 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:07.300119 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62ac4e5c-8f17-400e-bd44-026f3c769e90-console-oauth-config\") pod \"console-7f7fcd76f7-q7spb\" (UID: \"62ac4e5c-8f17-400e-bd44-026f3c769e90\") " pod="openshift-console/console-7f7fcd76f7-q7spb" Aug 04 09:42:07.300252 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:07.300183 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62ac4e5c-8f17-400e-bd44-026f3c769e90-console-serving-cert\") pod \"console-7f7fcd76f7-q7spb\" (UID: \"62ac4e5c-8f17-400e-bd44-026f3c769e90\") " pod="openshift-console/console-7f7fcd76f7-q7spb" Aug 04 09:42:07.307241 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:07.307211 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t27z\" (UniqueName: \"kubernetes.io/projected/62ac4e5c-8f17-400e-bd44-026f3c769e90-kube-api-access-9t27z\") pod \"console-7f7fcd76f7-q7spb\" (UID: \"62ac4e5c-8f17-400e-bd44-026f3c769e90\") " pod="openshift-console/console-7f7fcd76f7-q7spb" Aug 04 09:42:07.324944 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:07.324899 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f7fcd76f7-q7spb" Aug 04 09:42:07.481519 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:07.481493 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f7fcd76f7-q7spb"] Aug 04 09:42:07.483783 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:42:07.483753 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62ac4e5c_8f17_400e_bd44_026f3c769e90.slice/crio-0bbe6281bfc562b7c9071ac7692626a85a9742095112524387dbfa389a0cbf76 WatchSource:0}: Error finding container 0bbe6281bfc562b7c9071ac7692626a85a9742095112524387dbfa389a0cbf76: Status 404 returned error can't find the container with id 0bbe6281bfc562b7c9071ac7692626a85a9742095112524387dbfa389a0cbf76 Aug 04 09:42:08.291072 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:08.291031 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f7fcd76f7-q7spb" event={"ID":"62ac4e5c-8f17-400e-bd44-026f3c769e90","Type":"ContainerStarted","Data":"0bbe6281bfc562b7c9071ac7692626a85a9742095112524387dbfa389a0cbf76"} Aug 04 09:42:10.178765 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:10.178729 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:10.276578 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:10.276542 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:11.306952 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:11.306911 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f7fcd76f7-q7spb" event={"ID":"62ac4e5c-8f17-400e-bd44-026f3c769e90","Type":"ContainerStarted","Data":"96db7603a32b6c5fa6b3898601782ac573987f102ea9045b198b37f1e4a1da7a"} Aug 04 09:42:11.332921 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:11.332861 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f7fcd76f7-q7spb" podStartSLOduration=2.353055877 podStartE2EDuration="5.332835363s" totalImagesPullingTime="2.979779486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:42:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:42:11.330835053 +0000 UTC m=+100.074969472" watchObservedRunningTime="2026-08-04 09:42:11.332835363 +0000 UTC m=+100.076969783" Aug 04 09:42:13.939238 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:13.939202 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:14.040335 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:14.040302 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:14.143165 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:14.143134 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qcql5" Aug 04 09:42:14.168847 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:14.168791 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:14.262936 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:14.262897 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:17.326171 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:17.326124 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f7fcd76f7-q7spb" Aug 04 09:42:17.326581 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:17.326231 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f7fcd76f7-q7spb" Aug 04 09:42:17.331697 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:17.331664 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f7fcd76f7-q7spb" Aug 04 09:42:17.336269 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:17.336228 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f7fcd76f7-q7spb" Aug 04 09:42:17.923640 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:17.923597 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" Aug 04 09:42:17.923848 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:17.923675 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" Aug 04 09:42:19.341821 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:19.341755 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-77c974ffcf-bg556" event={"ID":"8304a19e-b591-4a42-aecb-13e8bc84f5b9","Type":"ContainerStarted","Data":"d2d1c09980449cc40939de284b13bb9bd1f41209e72ed0917c20b9aa04d2b7ce"} Aug 04 09:42:19.342310 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:19.341939 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-77c974ffcf-bg556" Aug 04 09:42:19.355572 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:19.355540 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-77c974ffcf-bg556" Aug 04 09:42:19.368843 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:19.368735 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-77c974ffcf-bg556" podStartSLOduration=1.522119387 podStartE2EDuration="18.368715832s" totalImagesPullingTime="16.846596445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:42:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:42:19.367638345 +0000 UTC m=+108.111772778" watchObservedRunningTime="2026-08-04 09:42:19.368715832 +0000 UTC m=+108.112850253" Aug 04 09:42:21.820648 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:21.820609 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:21.888409 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:21.888363 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:22.455658 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:22.455617 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:22.593768 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:22.593727 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:22.700533 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:22.700495 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:22.779420 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:22.779337 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:25.533512 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:25.533482 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f7fcd76f7-q7spb"] Aug 04 09:42:25.961495 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:25.961458 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:31.677159 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:31.677118 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:31.782216 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:31.782181 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:37.928432 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:37.928400 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" Aug 04 09:42:37.932406 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:37.932381 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-757d6d8489-6kbhk" Aug 04 09:42:43.904134 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:43.904098 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:44.006372 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:44.006337 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:42:50.555947 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:50.555903 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7f7fcd76f7-q7spb" podUID="62ac4e5c-8f17-400e-bd44-026f3c769e90" containerName="console" containerID="cri-o://96db7603a32b6c5fa6b3898601782ac573987f102ea9045b198b37f1e4a1da7a" gracePeriod=15 Aug 04 09:42:50.841053 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:50.840975 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f7fcd76f7-q7spb" Aug 04 09:42:50.895989 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:50.895949 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62ac4e5c-8f17-400e-bd44-026f3c769e90-service-ca\") pod \"62ac4e5c-8f17-400e-bd44-026f3c769e90\" (UID: \"62ac4e5c-8f17-400e-bd44-026f3c769e90\") " Aug 04 09:42:50.896160 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:50.896002 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62ac4e5c-8f17-400e-bd44-026f3c769e90-oauth-serving-cert\") pod \"62ac4e5c-8f17-400e-bd44-026f3c769e90\" (UID: \"62ac4e5c-8f17-400e-bd44-026f3c769e90\") " Aug 04 09:42:50.896160 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:50.896118 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62ac4e5c-8f17-400e-bd44-026f3c769e90-console-serving-cert\") pod \"62ac4e5c-8f17-400e-bd44-026f3c769e90\" (UID: \"62ac4e5c-8f17-400e-bd44-026f3c769e90\") " Aug 04 09:42:50.896278 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:50.896203 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62ac4e5c-8f17-400e-bd44-026f3c769e90-console-oauth-config\") pod \"62ac4e5c-8f17-400e-bd44-026f3c769e90\" (UID: \"62ac4e5c-8f17-400e-bd44-026f3c769e90\") " Aug 04 09:42:50.896278 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:50.896236 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62ac4e5c-8f17-400e-bd44-026f3c769e90-console-config\") pod \"62ac4e5c-8f17-400e-bd44-026f3c769e90\" (UID: \"62ac4e5c-8f17-400e-bd44-026f3c769e90\") " Aug 04 09:42:50.896278 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:50.896257 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9t27z\" (UniqueName: \"kubernetes.io/projected/62ac4e5c-8f17-400e-bd44-026f3c769e90-kube-api-access-9t27z\") pod \"62ac4e5c-8f17-400e-bd44-026f3c769e90\" (UID: \"62ac4e5c-8f17-400e-bd44-026f3c769e90\") " Aug 04 09:42:50.896454 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:50.896426 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62ac4e5c-8f17-400e-bd44-026f3c769e90-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "62ac4e5c-8f17-400e-bd44-026f3c769e90" (UID: "62ac4e5c-8f17-400e-bd44-026f3c769e90"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:42:50.896583 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:50.896446 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62ac4e5c-8f17-400e-bd44-026f3c769e90-service-ca" (OuterVolumeSpecName: "service-ca") pod "62ac4e5c-8f17-400e-bd44-026f3c769e90" (UID: "62ac4e5c-8f17-400e-bd44-026f3c769e90"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:42:50.896695 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:50.896503 2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62ac4e5c-8f17-400e-bd44-026f3c769e90-oauth-serving-cert\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:42:50.896695 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:50.896625 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62ac4e5c-8f17-400e-bd44-026f3c769e90-console-config" (OuterVolumeSpecName: "console-config") pod "62ac4e5c-8f17-400e-bd44-026f3c769e90" (UID: "62ac4e5c-8f17-400e-bd44-026f3c769e90"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:42:50.898393 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:50.898366 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62ac4e5c-8f17-400e-bd44-026f3c769e90-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "62ac4e5c-8f17-400e-bd44-026f3c769e90" (UID: "62ac4e5c-8f17-400e-bd44-026f3c769e90"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:42:50.898493 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:50.898414 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62ac4e5c-8f17-400e-bd44-026f3c769e90-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "62ac4e5c-8f17-400e-bd44-026f3c769e90" (UID: "62ac4e5c-8f17-400e-bd44-026f3c769e90"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:42:50.898535 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:50.898499 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62ac4e5c-8f17-400e-bd44-026f3c769e90-kube-api-access-9t27z" (OuterVolumeSpecName: "kube-api-access-9t27z") pod "62ac4e5c-8f17-400e-bd44-026f3c769e90" (UID: "62ac4e5c-8f17-400e-bd44-026f3c769e90"). InnerVolumeSpecName "kube-api-access-9t27z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:42:50.997012 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:50.996970 2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62ac4e5c-8f17-400e-bd44-026f3c769e90-service-ca\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:42:50.997012 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:50.997003 2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62ac4e5c-8f17-400e-bd44-026f3c769e90-console-serving-cert\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:42:50.997012 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:50.997013 2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62ac4e5c-8f17-400e-bd44-026f3c769e90-console-oauth-config\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:42:50.997012 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:50.997022 2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62ac4e5c-8f17-400e-bd44-026f3c769e90-console-config\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:42:50.997267 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:50.997031 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9t27z\" (UniqueName: \"kubernetes.io/projected/62ac4e5c-8f17-400e-bd44-026f3c769e90-kube-api-access-9t27z\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:42:51.467808 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:51.467756 2572 generic.go:356] "Generic (PLEG): container finished" podID="62ac4e5c-8f17-400e-bd44-026f3c769e90" containerID="96db7603a32b6c5fa6b3898601782ac573987f102ea9045b198b37f1e4a1da7a" exitCode=0 Aug 04 09:42:51.467981 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:51.467813 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f7fcd76f7-q7spb" Aug 04 09:42:51.467981 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:51.467845 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f7fcd76f7-q7spb" event={"ID":"62ac4e5c-8f17-400e-bd44-026f3c769e90","Type":"ContainerDied","Data":"96db7603a32b6c5fa6b3898601782ac573987f102ea9045b198b37f1e4a1da7a"} Aug 04 09:42:51.467981 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:51.467900 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f7fcd76f7-q7spb" event={"ID":"62ac4e5c-8f17-400e-bd44-026f3c769e90","Type":"ContainerDied","Data":"0bbe6281bfc562b7c9071ac7692626a85a9742095112524387dbfa389a0cbf76"} Aug 04 09:42:51.467981 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:51.467936 2572 scope.go:117] "RemoveContainer" containerID="96db7603a32b6c5fa6b3898601782ac573987f102ea9045b198b37f1e4a1da7a" Aug 04 09:42:51.476928 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:51.476910 2572 scope.go:117] "RemoveContainer" containerID="96db7603a32b6c5fa6b3898601782ac573987f102ea9045b198b37f1e4a1da7a" Aug 04 09:42:51.477250 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:42:51.477230 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96db7603a32b6c5fa6b3898601782ac573987f102ea9045b198b37f1e4a1da7a\": container with ID starting with 96db7603a32b6c5fa6b3898601782ac573987f102ea9045b198b37f1e4a1da7a not found: ID does not exist" containerID="96db7603a32b6c5fa6b3898601782ac573987f102ea9045b198b37f1e4a1da7a" Aug 04 09:42:51.477316 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:51.477261 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96db7603a32b6c5fa6b3898601782ac573987f102ea9045b198b37f1e4a1da7a"} err="failed to get container status \"96db7603a32b6c5fa6b3898601782ac573987f102ea9045b198b37f1e4a1da7a\": rpc error: code = NotFound desc = could not find container \"96db7603a32b6c5fa6b3898601782ac573987f102ea9045b198b37f1e4a1da7a\": container with ID starting with 96db7603a32b6c5fa6b3898601782ac573987f102ea9045b198b37f1e4a1da7a not found: ID does not exist" Aug 04 09:42:51.496359 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:51.496324 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f7fcd76f7-q7spb"] Aug 04 09:42:51.502434 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:51.502406 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f7fcd76f7-q7spb"] Aug 04 09:42:51.782618 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:42:51.782537 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62ac4e5c-8f17-400e-bd44-026f3c769e90" path="/var/lib/kubelet/pods/62ac4e5c-8f17-400e-bd44-026f3c769e90/volumes" Aug 04 09:43:05.353553 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:05.353478 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:43:09.377418 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:09.377382 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:43:09.473792 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:09.473757 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:43:13.736746 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.736706 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-689b989dbb-xjpfk"] Aug 04 09:43:13.737219 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.737047 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62ac4e5c-8f17-400e-bd44-026f3c769e90" containerName="console" Aug 04 09:43:13.737219 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.737060 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="62ac4e5c-8f17-400e-bd44-026f3c769e90" containerName="console" Aug 04 09:43:13.737219 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.737132 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="62ac4e5c-8f17-400e-bd44-026f3c769e90" containerName="console" Aug 04 09:43:13.740262 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.740240 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-689b989dbb-xjpfk" Aug 04 09:43:13.743572 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.743550 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-8x9rc\"" Aug 04 09:43:13.744253 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.744226 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Aug 04 09:43:13.744404 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.744297 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Aug 04 09:43:13.744404 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.744301 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Aug 04 09:43:13.744834 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.744817 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Aug 04 09:43:13.745598 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.745584 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Aug 04 09:43:13.763787 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.763757 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Aug 04 09:43:13.767685 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.767650 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-689b989dbb-xjpfk"] Aug 04 09:43:13.769159 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.769134 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 04 09:43:13.769495 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.769465 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerName="alertmanager" containerID="cri-o://8f23cd1f0e7b8e087b3de416f2284535a5b1e5236a0430b8f8982610a6066cab" gracePeriod=120 Aug 04 09:43:13.769636 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.769492 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerName="prom-label-proxy" containerID="cri-o://1839c183a96aebc21fc547594fefe37f11ce0cecef1877f6396ae47c14a7187a" gracePeriod=120 Aug 04 09:43:13.769636 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.769488 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerName="kube-rbac-proxy" containerID="cri-o://cd14f36db6dd8646424ebcd7cd6e8f2de489f47d7615089be7769a44440ce18b" gracePeriod=120 Aug 04 09:43:13.769636 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.769503 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerName="kube-rbac-proxy-metric" containerID="cri-o://933639ee0fcafa4406f522e098f1a38b24dda4420d63c49c8dd5be616cef0fa2" gracePeriod=120 Aug 04 09:43:13.769636 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.769529 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerName="config-reloader" containerID="cri-o://2058ff56324d4814eba7505df61abbd33ef778a1c4853349732031ad9ba937f1" gracePeriod=120 Aug 04 09:43:13.769636 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.769515 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerName="kube-rbac-proxy-web" containerID="cri-o://8b70ccb8a6557d65e7319700e015a35ee0c86387beabeb4aa491b7e8accfce70" gracePeriod=120 Aug 04 09:43:13.787196 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.786821 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp9ch\" (UniqueName: \"kubernetes.io/projected/5a805a34-24d3-4538-81a4-65aed5e764dc-kube-api-access-bp9ch\") pod \"console-689b989dbb-xjpfk\" (UID: \"5a805a34-24d3-4538-81a4-65aed5e764dc\") " pod="openshift-console/console-689b989dbb-xjpfk" Aug 04 09:43:13.787196 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.786874 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a805a34-24d3-4538-81a4-65aed5e764dc-oauth-serving-cert\") pod \"console-689b989dbb-xjpfk\" (UID: \"5a805a34-24d3-4538-81a4-65aed5e764dc\") " pod="openshift-console/console-689b989dbb-xjpfk" Aug 04 09:43:13.787196 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.786948 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a805a34-24d3-4538-81a4-65aed5e764dc-console-oauth-config\") pod \"console-689b989dbb-xjpfk\" (UID: \"5a805a34-24d3-4538-81a4-65aed5e764dc\") " pod="openshift-console/console-689b989dbb-xjpfk" Aug 04 09:43:13.787196 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.787003 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a805a34-24d3-4538-81a4-65aed5e764dc-console-serving-cert\") pod \"console-689b989dbb-xjpfk\" (UID: \"5a805a34-24d3-4538-81a4-65aed5e764dc\") " pod="openshift-console/console-689b989dbb-xjpfk" Aug 04 09:43:13.787196 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.787036 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a805a34-24d3-4538-81a4-65aed5e764dc-service-ca\") pod \"console-689b989dbb-xjpfk\" (UID: \"5a805a34-24d3-4538-81a4-65aed5e764dc\") " pod="openshift-console/console-689b989dbb-xjpfk" Aug 04 09:43:13.787196 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.787065 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a805a34-24d3-4538-81a4-65aed5e764dc-console-config\") pod \"console-689b989dbb-xjpfk\" (UID: \"5a805a34-24d3-4538-81a4-65aed5e764dc\") " pod="openshift-console/console-689b989dbb-xjpfk" Aug 04 09:43:13.787196 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.787102 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a805a34-24d3-4538-81a4-65aed5e764dc-trusted-ca-bundle\") pod \"console-689b989dbb-xjpfk\" (UID: \"5a805a34-24d3-4538-81a4-65aed5e764dc\") " pod="openshift-console/console-689b989dbb-xjpfk" Aug 04 09:43:13.814384 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.814350 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:43:13.887692 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.887665 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a805a34-24d3-4538-81a4-65aed5e764dc-console-serving-cert\") pod \"console-689b989dbb-xjpfk\" (UID: \"5a805a34-24d3-4538-81a4-65aed5e764dc\") " pod="openshift-console/console-689b989dbb-xjpfk" Aug 04 09:43:13.887843 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.887701 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a805a34-24d3-4538-81a4-65aed5e764dc-service-ca\") pod \"console-689b989dbb-xjpfk\" (UID: \"5a805a34-24d3-4538-81a4-65aed5e764dc\") " pod="openshift-console/console-689b989dbb-xjpfk" Aug 04 09:43:13.887843 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.887726 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a805a34-24d3-4538-81a4-65aed5e764dc-console-config\") pod \"console-689b989dbb-xjpfk\" (UID: \"5a805a34-24d3-4538-81a4-65aed5e764dc\") " pod="openshift-console/console-689b989dbb-xjpfk" Aug 04 09:43:13.887843 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.887751 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a805a34-24d3-4538-81a4-65aed5e764dc-trusted-ca-bundle\") pod \"console-689b989dbb-xjpfk\" (UID: \"5a805a34-24d3-4538-81a4-65aed5e764dc\") " pod="openshift-console/console-689b989dbb-xjpfk" Aug 04 09:43:13.888015 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.887924 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp9ch\" (UniqueName: \"kubernetes.io/projected/5a805a34-24d3-4538-81a4-65aed5e764dc-kube-api-access-bp9ch\") pod \"console-689b989dbb-xjpfk\" (UID: \"5a805a34-24d3-4538-81a4-65aed5e764dc\") " pod="openshift-console/console-689b989dbb-xjpfk" Aug 04 09:43:13.888015 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.887975 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a805a34-24d3-4538-81a4-65aed5e764dc-oauth-serving-cert\") pod \"console-689b989dbb-xjpfk\" (UID: \"5a805a34-24d3-4538-81a4-65aed5e764dc\") " pod="openshift-console/console-689b989dbb-xjpfk" Aug 04 09:43:13.888112 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.888064 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a805a34-24d3-4538-81a4-65aed5e764dc-console-oauth-config\") pod \"console-689b989dbb-xjpfk\" (UID: \"5a805a34-24d3-4538-81a4-65aed5e764dc\") " pod="openshift-console/console-689b989dbb-xjpfk" Aug 04 09:43:13.888491 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.888468 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a805a34-24d3-4538-81a4-65aed5e764dc-console-config\") pod \"console-689b989dbb-xjpfk\" (UID: \"5a805a34-24d3-4538-81a4-65aed5e764dc\") " pod="openshift-console/console-689b989dbb-xjpfk" Aug 04 09:43:13.888603 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.888509 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a805a34-24d3-4538-81a4-65aed5e764dc-service-ca\") pod \"console-689b989dbb-xjpfk\" (UID: \"5a805a34-24d3-4538-81a4-65aed5e764dc\") " pod="openshift-console/console-689b989dbb-xjpfk" Aug 04 09:43:13.888660 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.888631 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a805a34-24d3-4538-81a4-65aed5e764dc-oauth-serving-cert\") pod \"console-689b989dbb-xjpfk\" (UID: \"5a805a34-24d3-4538-81a4-65aed5e764dc\") " pod="openshift-console/console-689b989dbb-xjpfk" Aug 04 09:43:13.888908 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.888888 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a805a34-24d3-4538-81a4-65aed5e764dc-trusted-ca-bundle\") pod \"console-689b989dbb-xjpfk\" (UID: \"5a805a34-24d3-4538-81a4-65aed5e764dc\") " pod="openshift-console/console-689b989dbb-xjpfk" Aug 04 09:43:13.890049 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.890022 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a805a34-24d3-4538-81a4-65aed5e764dc-console-serving-cert\") pod \"console-689b989dbb-xjpfk\" (UID: \"5a805a34-24d3-4538-81a4-65aed5e764dc\") " pod="openshift-console/console-689b989dbb-xjpfk" Aug 04 09:43:13.890397 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.890377 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a805a34-24d3-4538-81a4-65aed5e764dc-console-oauth-config\") pod \"console-689b989dbb-xjpfk\" (UID: \"5a805a34-24d3-4538-81a4-65aed5e764dc\") " pod="openshift-console/console-689b989dbb-xjpfk" Aug 04 09:43:13.894757 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.894734 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:43:13.899086 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:13.899063 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp9ch\" (UniqueName: \"kubernetes.io/projected/5a805a34-24d3-4538-81a4-65aed5e764dc-kube-api-access-bp9ch\") pod \"console-689b989dbb-xjpfk\" (UID: \"5a805a34-24d3-4538-81a4-65aed5e764dc\") " pod="openshift-console/console-689b989dbb-xjpfk" Aug 04 09:43:14.049576 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:14.049487 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-689b989dbb-xjpfk" Aug 04 09:43:14.180829 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:14.180788 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-689b989dbb-xjpfk"] Aug 04 09:43:14.183418 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:43:14.183390 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a805a34_24d3_4538_81a4_65aed5e764dc.slice/crio-bab2e65c6612a7e5885fa2164c05e61b760b2512c7580076ea19cc3400cf4d97 WatchSource:0}: Error finding container bab2e65c6612a7e5885fa2164c05e61b760b2512c7580076ea19cc3400cf4d97: Status 404 returned error can't find the container with id bab2e65c6612a7e5885fa2164c05e61b760b2512c7580076ea19cc3400cf4d97 Aug 04 09:43:14.572899 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:14.572861 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-689b989dbb-xjpfk" event={"ID":"5a805a34-24d3-4538-81a4-65aed5e764dc","Type":"ContainerStarted","Data":"4aa09d498515c0ff4950b2bf87c5f66235c899f552e5fe9fd3ae7db738f08fca"} Aug 04 09:43:14.573118 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:14.572944 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-689b989dbb-xjpfk" event={"ID":"5a805a34-24d3-4538-81a4-65aed5e764dc","Type":"ContainerStarted","Data":"bab2e65c6612a7e5885fa2164c05e61b760b2512c7580076ea19cc3400cf4d97"} Aug 04 09:43:14.576217 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:14.576191 2572 generic.go:356] "Generic (PLEG): container finished" podID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerID="1839c183a96aebc21fc547594fefe37f11ce0cecef1877f6396ae47c14a7187a" exitCode=0 Aug 04 09:43:14.576217 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:14.576211 2572 generic.go:356] "Generic (PLEG): container finished" podID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerID="cd14f36db6dd8646424ebcd7cd6e8f2de489f47d7615089be7769a44440ce18b" exitCode=0 Aug 04 09:43:14.576217 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:14.576218 2572 generic.go:356] "Generic (PLEG): container finished" podID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerID="2058ff56324d4814eba7505df61abbd33ef778a1c4853349732031ad9ba937f1" exitCode=0 Aug 04 09:43:14.576217 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:14.576223 2572 generic.go:356] "Generic (PLEG): container finished" podID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerID="8f23cd1f0e7b8e087b3de416f2284535a5b1e5236a0430b8f8982610a6066cab" exitCode=0 Aug 04 09:43:14.576425 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:14.576257 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"59fb9607-ffdc-4604-82eb-46b1187f6628","Type":"ContainerDied","Data":"1839c183a96aebc21fc547594fefe37f11ce0cecef1877f6396ae47c14a7187a"} Aug 04 09:43:14.576425 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:14.576314 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"59fb9607-ffdc-4604-82eb-46b1187f6628","Type":"ContainerDied","Data":"cd14f36db6dd8646424ebcd7cd6e8f2de489f47d7615089be7769a44440ce18b"} Aug 04 09:43:14.576425 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:14.576340 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"59fb9607-ffdc-4604-82eb-46b1187f6628","Type":"ContainerDied","Data":"2058ff56324d4814eba7505df61abbd33ef778a1c4853349732031ad9ba937f1"} Aug 04 09:43:14.576425 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:14.576364 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"59fb9607-ffdc-4604-82eb-46b1187f6628","Type":"ContainerDied","Data":"8f23cd1f0e7b8e087b3de416f2284535a5b1e5236a0430b8f8982610a6066cab"} Aug 04 09:43:14.597865 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:14.597790 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-689b989dbb-xjpfk" podStartSLOduration=1.5977756090000002 podStartE2EDuration="1.597775609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:43:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:43:14.597654433 +0000 UTC m=+163.341788853" watchObservedRunningTime="2026-08-04 09:43:14.597775609 +0000 UTC m=+163.341910028" Aug 04 09:43:15.007531 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.007502 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.101036 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.100943 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48rqz\" (UniqueName: \"kubernetes.io/projected/59fb9607-ffdc-4604-82eb-46b1187f6628-kube-api-access-48rqz\") pod \"59fb9607-ffdc-4604-82eb-46b1187f6628\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " Aug 04 09:43:15.101036 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.100999 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/59fb9607-ffdc-4604-82eb-46b1187f6628-config-out\") pod \"59fb9607-ffdc-4604-82eb-46b1187f6628\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " Aug 04 09:43:15.101036 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.101026 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-config-volume\") pod \"59fb9607-ffdc-4604-82eb-46b1187f6628\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " Aug 04 09:43:15.101382 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.101130 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59fb9607-ffdc-4604-82eb-46b1187f6628-metrics-client-ca\") pod \"59fb9607-ffdc-4604-82eb-46b1187f6628\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " Aug 04 09:43:15.101382 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.101172 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-cluster-tls-config\") pod \"59fb9607-ffdc-4604-82eb-46b1187f6628\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " Aug 04 09:43:15.101382 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.101197 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/59fb9607-ffdc-4604-82eb-46b1187f6628-tls-assets\") pod \"59fb9607-ffdc-4604-82eb-46b1187f6628\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " Aug 04 09:43:15.101382 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.101331 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-secret-alertmanager-kube-rbac-proxy-web\") pod \"59fb9607-ffdc-4604-82eb-46b1187f6628\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " Aug 04 09:43:15.101583 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.101386 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-secret-alertmanager-kube-rbac-proxy\") pod \"59fb9607-ffdc-4604-82eb-46b1187f6628\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " Aug 04 09:43:15.101583 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.101449 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-secret-alertmanager-kube-rbac-proxy-metric\") pod \"59fb9607-ffdc-4604-82eb-46b1187f6628\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " Aug 04 09:43:15.101583 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.101482 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59fb9607-ffdc-4604-82eb-46b1187f6628-alertmanager-trusted-ca-bundle\") pod \"59fb9607-ffdc-4604-82eb-46b1187f6628\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " Aug 04 09:43:15.101583 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.101519 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-secret-alertmanager-main-tls\") pod \"59fb9607-ffdc-4604-82eb-46b1187f6628\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " Aug 04 09:43:15.101583 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.101533 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59fb9607-ffdc-4604-82eb-46b1187f6628-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "59fb9607-ffdc-4604-82eb-46b1187f6628" (UID: "59fb9607-ffdc-4604-82eb-46b1187f6628"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:43:15.101583 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.101567 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-web-config\") pod \"59fb9607-ffdc-4604-82eb-46b1187f6628\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " Aug 04 09:43:15.101891 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.101602 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/59fb9607-ffdc-4604-82eb-46b1187f6628-alertmanager-main-db\") pod \"59fb9607-ffdc-4604-82eb-46b1187f6628\" (UID: \"59fb9607-ffdc-4604-82eb-46b1187f6628\") " Aug 04 09:43:15.101942 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.101903 2572 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/59fb9607-ffdc-4604-82eb-46b1187f6628-metrics-client-ca\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:43:15.102023 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.101995 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59fb9607-ffdc-4604-82eb-46b1187f6628-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "59fb9607-ffdc-4604-82eb-46b1187f6628" (UID: "59fb9607-ffdc-4604-82eb-46b1187f6628"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:43:15.102609 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.102229 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59fb9607-ffdc-4604-82eb-46b1187f6628-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "59fb9607-ffdc-4604-82eb-46b1187f6628" (UID: "59fb9607-ffdc-4604-82eb-46b1187f6628"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:43:15.104281 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.104250 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59fb9607-ffdc-4604-82eb-46b1187f6628-kube-api-access-48rqz" (OuterVolumeSpecName: "kube-api-access-48rqz") pod "59fb9607-ffdc-4604-82eb-46b1187f6628" (UID: "59fb9607-ffdc-4604-82eb-46b1187f6628"). InnerVolumeSpecName "kube-api-access-48rqz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:43:15.104596 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.104573 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59fb9607-ffdc-4604-82eb-46b1187f6628-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "59fb9607-ffdc-4604-82eb-46b1187f6628" (UID: "59fb9607-ffdc-4604-82eb-46b1187f6628"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:43:15.104672 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.104637 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-config-volume" (OuterVolumeSpecName: "config-volume") pod "59fb9607-ffdc-4604-82eb-46b1187f6628" (UID: "59fb9607-ffdc-4604-82eb-46b1187f6628"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:43:15.104732 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.104692 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "59fb9607-ffdc-4604-82eb-46b1187f6628" (UID: "59fb9607-ffdc-4604-82eb-46b1187f6628"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:43:15.104915 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.104888 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "59fb9607-ffdc-4604-82eb-46b1187f6628" (UID: "59fb9607-ffdc-4604-82eb-46b1187f6628"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:43:15.105056 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.105001 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59fb9607-ffdc-4604-82eb-46b1187f6628-config-out" (OuterVolumeSpecName: "config-out") pod "59fb9607-ffdc-4604-82eb-46b1187f6628" (UID: "59fb9607-ffdc-4604-82eb-46b1187f6628"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:43:15.105350 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.105326 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "59fb9607-ffdc-4604-82eb-46b1187f6628" (UID: "59fb9607-ffdc-4604-82eb-46b1187f6628"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:43:15.105427 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.105401 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "59fb9607-ffdc-4604-82eb-46b1187f6628" (UID: "59fb9607-ffdc-4604-82eb-46b1187f6628"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:43:15.108434 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.108409 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "59fb9607-ffdc-4604-82eb-46b1187f6628" (UID: "59fb9607-ffdc-4604-82eb-46b1187f6628"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:43:15.114958 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.114791 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-web-config" (OuterVolumeSpecName: "web-config") pod "59fb9607-ffdc-4604-82eb-46b1187f6628" (UID: "59fb9607-ffdc-4604-82eb-46b1187f6628"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:43:15.202352 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.202300 2572 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/59fb9607-ffdc-4604-82eb-46b1187f6628-config-out\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:43:15.202352 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.202344 2572 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-config-volume\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:43:15.202352 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.202354 2572 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-cluster-tls-config\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:43:15.202352 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.202363 2572 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/59fb9607-ffdc-4604-82eb-46b1187f6628-tls-assets\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:43:15.202352 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.202373 2572 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:43:15.202654 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.202383 2572 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:43:15.202654 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.202391 2572 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:43:15.202654 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.202402 2572 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59fb9607-ffdc-4604-82eb-46b1187f6628-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:43:15.202654 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.202410 2572 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-secret-alertmanager-main-tls\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:43:15.202654 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.202419 2572 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/59fb9607-ffdc-4604-82eb-46b1187f6628-web-config\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:43:15.202654 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.202427 2572 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/59fb9607-ffdc-4604-82eb-46b1187f6628-alertmanager-main-db\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:43:15.202654 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.202434 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48rqz\" (UniqueName: \"kubernetes.io/projected/59fb9607-ffdc-4604-82eb-46b1187f6628-kube-api-access-48rqz\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:43:15.583460 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.583420 2572 generic.go:356] "Generic (PLEG): container finished" podID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerID="933639ee0fcafa4406f522e098f1a38b24dda4420d63c49c8dd5be616cef0fa2" exitCode=0 Aug 04 09:43:15.583460 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.583446 2572 generic.go:356] "Generic (PLEG): container finished" podID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerID="8b70ccb8a6557d65e7319700e015a35ee0c86387beabeb4aa491b7e8accfce70" exitCode=0 Aug 04 09:43:15.583725 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.583497 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.583725 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.583496 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"59fb9607-ffdc-4604-82eb-46b1187f6628","Type":"ContainerDied","Data":"933639ee0fcafa4406f522e098f1a38b24dda4420d63c49c8dd5be616cef0fa2"} Aug 04 09:43:15.583725 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.583574 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"59fb9607-ffdc-4604-82eb-46b1187f6628","Type":"ContainerDied","Data":"8b70ccb8a6557d65e7319700e015a35ee0c86387beabeb4aa491b7e8accfce70"} Aug 04 09:43:15.583725 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.583604 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"59fb9607-ffdc-4604-82eb-46b1187f6628","Type":"ContainerDied","Data":"d8b14bb90244e388630f212c2714535c75ca0198fdff1302f433d168b606928e"} Aug 04 09:43:15.583725 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.583635 2572 scope.go:117] "RemoveContainer" containerID="1839c183a96aebc21fc547594fefe37f11ce0cecef1877f6396ae47c14a7187a" Aug 04 09:43:15.592173 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.592150 2572 scope.go:117] "RemoveContainer" containerID="933639ee0fcafa4406f522e098f1a38b24dda4420d63c49c8dd5be616cef0fa2" Aug 04 09:43:15.599309 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.599288 2572 scope.go:117] "RemoveContainer" containerID="cd14f36db6dd8646424ebcd7cd6e8f2de489f47d7615089be7769a44440ce18b" Aug 04 09:43:15.606283 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.606263 2572 scope.go:117] "RemoveContainer" containerID="8b70ccb8a6557d65e7319700e015a35ee0c86387beabeb4aa491b7e8accfce70" Aug 04 09:43:15.611815 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.611768 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 04 09:43:15.614317 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.614290 2572 scope.go:117] "RemoveContainer" containerID="2058ff56324d4814eba7505df61abbd33ef778a1c4853349732031ad9ba937f1" Aug 04 09:43:15.616221 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.616199 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 04 09:43:15.621876 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.621852 2572 scope.go:117] "RemoveContainer" containerID="8f23cd1f0e7b8e087b3de416f2284535a5b1e5236a0430b8f8982610a6066cab" Aug 04 09:43:15.629270 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.629251 2572 scope.go:117] "RemoveContainer" containerID="48be1d4a8fd45aba311286adc7583d434189c35f4e769e31fd109ee1b9f0634f" Aug 04 09:43:15.636536 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.636514 2572 scope.go:117] "RemoveContainer" containerID="1839c183a96aebc21fc547594fefe37f11ce0cecef1877f6396ae47c14a7187a" Aug 04 09:43:15.636853 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:43:15.636830 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1839c183a96aebc21fc547594fefe37f11ce0cecef1877f6396ae47c14a7187a\": container with ID starting with 1839c183a96aebc21fc547594fefe37f11ce0cecef1877f6396ae47c14a7187a not found: ID does not exist" containerID="1839c183a96aebc21fc547594fefe37f11ce0cecef1877f6396ae47c14a7187a" Aug 04 09:43:15.636951 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.636861 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1839c183a96aebc21fc547594fefe37f11ce0cecef1877f6396ae47c14a7187a"} err="failed to get container status \"1839c183a96aebc21fc547594fefe37f11ce0cecef1877f6396ae47c14a7187a\": rpc error: code = NotFound desc = could not find container \"1839c183a96aebc21fc547594fefe37f11ce0cecef1877f6396ae47c14a7187a\": container with ID starting with 1839c183a96aebc21fc547594fefe37f11ce0cecef1877f6396ae47c14a7187a not found: ID does not exist" Aug 04 09:43:15.636951 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.636883 2572 scope.go:117] "RemoveContainer" containerID="933639ee0fcafa4406f522e098f1a38b24dda4420d63c49c8dd5be616cef0fa2" Aug 04 09:43:15.637146 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:43:15.637126 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"933639ee0fcafa4406f522e098f1a38b24dda4420d63c49c8dd5be616cef0fa2\": container with ID starting with 933639ee0fcafa4406f522e098f1a38b24dda4420d63c49c8dd5be616cef0fa2 not found: ID does not exist" containerID="933639ee0fcafa4406f522e098f1a38b24dda4420d63c49c8dd5be616cef0fa2" Aug 04 09:43:15.637185 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.637150 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"933639ee0fcafa4406f522e098f1a38b24dda4420d63c49c8dd5be616cef0fa2"} err="failed to get container status \"933639ee0fcafa4406f522e098f1a38b24dda4420d63c49c8dd5be616cef0fa2\": rpc error: code = NotFound desc = could not find container \"933639ee0fcafa4406f522e098f1a38b24dda4420d63c49c8dd5be616cef0fa2\": container with ID starting with 933639ee0fcafa4406f522e098f1a38b24dda4420d63c49c8dd5be616cef0fa2 not found: ID does not exist" Aug 04 09:43:15.637185 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.637164 2572 scope.go:117] "RemoveContainer" containerID="cd14f36db6dd8646424ebcd7cd6e8f2de489f47d7615089be7769a44440ce18b" Aug 04 09:43:15.637363 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:43:15.637349 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd14f36db6dd8646424ebcd7cd6e8f2de489f47d7615089be7769a44440ce18b\": container with ID starting with cd14f36db6dd8646424ebcd7cd6e8f2de489f47d7615089be7769a44440ce18b not found: ID does not exist" containerID="cd14f36db6dd8646424ebcd7cd6e8f2de489f47d7615089be7769a44440ce18b" Aug 04 09:43:15.637400 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.637366 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd14f36db6dd8646424ebcd7cd6e8f2de489f47d7615089be7769a44440ce18b"} err="failed to get container status \"cd14f36db6dd8646424ebcd7cd6e8f2de489f47d7615089be7769a44440ce18b\": rpc error: code = NotFound desc = could not find container \"cd14f36db6dd8646424ebcd7cd6e8f2de489f47d7615089be7769a44440ce18b\": container with ID starting with cd14f36db6dd8646424ebcd7cd6e8f2de489f47d7615089be7769a44440ce18b not found: ID does not exist" Aug 04 09:43:15.637400 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.637378 2572 scope.go:117] "RemoveContainer" containerID="8b70ccb8a6557d65e7319700e015a35ee0c86387beabeb4aa491b7e8accfce70" Aug 04 09:43:15.637561 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:43:15.637548 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b70ccb8a6557d65e7319700e015a35ee0c86387beabeb4aa491b7e8accfce70\": container with ID starting with 8b70ccb8a6557d65e7319700e015a35ee0c86387beabeb4aa491b7e8accfce70 not found: ID does not exist" containerID="8b70ccb8a6557d65e7319700e015a35ee0c86387beabeb4aa491b7e8accfce70" Aug 04 09:43:15.637602 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.637562 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b70ccb8a6557d65e7319700e015a35ee0c86387beabeb4aa491b7e8accfce70"} err="failed to get container status \"8b70ccb8a6557d65e7319700e015a35ee0c86387beabeb4aa491b7e8accfce70\": rpc error: code = NotFound desc = could not find container \"8b70ccb8a6557d65e7319700e015a35ee0c86387beabeb4aa491b7e8accfce70\": container with ID starting with 8b70ccb8a6557d65e7319700e015a35ee0c86387beabeb4aa491b7e8accfce70 not found: ID does not exist" Aug 04 09:43:15.637602 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.637573 2572 scope.go:117] "RemoveContainer" containerID="2058ff56324d4814eba7505df61abbd33ef778a1c4853349732031ad9ba937f1" Aug 04 09:43:15.637744 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:43:15.637730 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2058ff56324d4814eba7505df61abbd33ef778a1c4853349732031ad9ba937f1\": container with ID starting with 2058ff56324d4814eba7505df61abbd33ef778a1c4853349732031ad9ba937f1 not found: ID does not exist" containerID="2058ff56324d4814eba7505df61abbd33ef778a1c4853349732031ad9ba937f1" Aug 04 09:43:15.637785 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.637745 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2058ff56324d4814eba7505df61abbd33ef778a1c4853349732031ad9ba937f1"} err="failed to get container status \"2058ff56324d4814eba7505df61abbd33ef778a1c4853349732031ad9ba937f1\": rpc error: code = NotFound desc = could not find container \"2058ff56324d4814eba7505df61abbd33ef778a1c4853349732031ad9ba937f1\": container with ID starting with 2058ff56324d4814eba7505df61abbd33ef778a1c4853349732031ad9ba937f1 not found: ID does not exist" Aug 04 09:43:15.637785 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.637757 2572 scope.go:117] "RemoveContainer" containerID="8f23cd1f0e7b8e087b3de416f2284535a5b1e5236a0430b8f8982610a6066cab" Aug 04 09:43:15.637946 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:43:15.637932 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f23cd1f0e7b8e087b3de416f2284535a5b1e5236a0430b8f8982610a6066cab\": container with ID starting with 8f23cd1f0e7b8e087b3de416f2284535a5b1e5236a0430b8f8982610a6066cab not found: ID does not exist" containerID="8f23cd1f0e7b8e087b3de416f2284535a5b1e5236a0430b8f8982610a6066cab" Aug 04 09:43:15.637999 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.637948 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f23cd1f0e7b8e087b3de416f2284535a5b1e5236a0430b8f8982610a6066cab"} err="failed to get container status \"8f23cd1f0e7b8e087b3de416f2284535a5b1e5236a0430b8f8982610a6066cab\": rpc error: code = NotFound desc = could not find container \"8f23cd1f0e7b8e087b3de416f2284535a5b1e5236a0430b8f8982610a6066cab\": container with ID starting with 8f23cd1f0e7b8e087b3de416f2284535a5b1e5236a0430b8f8982610a6066cab not found: ID does not exist" Aug 04 09:43:15.637999 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.637959 2572 scope.go:117] "RemoveContainer" containerID="48be1d4a8fd45aba311286adc7583d434189c35f4e769e31fd109ee1b9f0634f" Aug 04 09:43:15.638203 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:43:15.638184 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48be1d4a8fd45aba311286adc7583d434189c35f4e769e31fd109ee1b9f0634f\": container with ID starting with 48be1d4a8fd45aba311286adc7583d434189c35f4e769e31fd109ee1b9f0634f not found: ID does not exist" containerID="48be1d4a8fd45aba311286adc7583d434189c35f4e769e31fd109ee1b9f0634f" Aug 04 09:43:15.638259 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.638212 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48be1d4a8fd45aba311286adc7583d434189c35f4e769e31fd109ee1b9f0634f"} err="failed to get container status \"48be1d4a8fd45aba311286adc7583d434189c35f4e769e31fd109ee1b9f0634f\": rpc error: code = NotFound desc = could not find container \"48be1d4a8fd45aba311286adc7583d434189c35f4e769e31fd109ee1b9f0634f\": container with ID starting with 48be1d4a8fd45aba311286adc7583d434189c35f4e769e31fd109ee1b9f0634f not found: ID does not exist" Aug 04 09:43:15.638259 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.638235 2572 scope.go:117] "RemoveContainer" containerID="1839c183a96aebc21fc547594fefe37f11ce0cecef1877f6396ae47c14a7187a" Aug 04 09:43:15.638433 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.638417 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1839c183a96aebc21fc547594fefe37f11ce0cecef1877f6396ae47c14a7187a"} err="failed to get container status \"1839c183a96aebc21fc547594fefe37f11ce0cecef1877f6396ae47c14a7187a\": rpc error: code = NotFound desc = could not find container \"1839c183a96aebc21fc547594fefe37f11ce0cecef1877f6396ae47c14a7187a\": container with ID starting with 1839c183a96aebc21fc547594fefe37f11ce0cecef1877f6396ae47c14a7187a not found: ID does not exist" Aug 04 09:43:15.638479 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.638435 2572 scope.go:117] "RemoveContainer" containerID="933639ee0fcafa4406f522e098f1a38b24dda4420d63c49c8dd5be616cef0fa2" Aug 04 09:43:15.638621 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.638603 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"933639ee0fcafa4406f522e098f1a38b24dda4420d63c49c8dd5be616cef0fa2"} err="failed to get container status \"933639ee0fcafa4406f522e098f1a38b24dda4420d63c49c8dd5be616cef0fa2\": rpc error: code = NotFound desc = could not find container \"933639ee0fcafa4406f522e098f1a38b24dda4420d63c49c8dd5be616cef0fa2\": container with ID starting with 933639ee0fcafa4406f522e098f1a38b24dda4420d63c49c8dd5be616cef0fa2 not found: ID does not exist" Aug 04 09:43:15.638684 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.638622 2572 scope.go:117] "RemoveContainer" containerID="cd14f36db6dd8646424ebcd7cd6e8f2de489f47d7615089be7769a44440ce18b" Aug 04 09:43:15.638847 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.638829 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd14f36db6dd8646424ebcd7cd6e8f2de489f47d7615089be7769a44440ce18b"} err="failed to get container status \"cd14f36db6dd8646424ebcd7cd6e8f2de489f47d7615089be7769a44440ce18b\": rpc error: code = NotFound desc = could not find container \"cd14f36db6dd8646424ebcd7cd6e8f2de489f47d7615089be7769a44440ce18b\": container with ID starting with cd14f36db6dd8646424ebcd7cd6e8f2de489f47d7615089be7769a44440ce18b not found: ID does not exist" Aug 04 09:43:15.638926 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.638848 2572 scope.go:117] "RemoveContainer" containerID="8b70ccb8a6557d65e7319700e015a35ee0c86387beabeb4aa491b7e8accfce70" Aug 04 09:43:15.639096 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.639074 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b70ccb8a6557d65e7319700e015a35ee0c86387beabeb4aa491b7e8accfce70"} err="failed to get container status \"8b70ccb8a6557d65e7319700e015a35ee0c86387beabeb4aa491b7e8accfce70\": rpc error: code = NotFound desc = could not find container \"8b70ccb8a6557d65e7319700e015a35ee0c86387beabeb4aa491b7e8accfce70\": container with ID starting with 8b70ccb8a6557d65e7319700e015a35ee0c86387beabeb4aa491b7e8accfce70 not found: ID does not exist" Aug 04 09:43:15.639096 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.639095 2572 scope.go:117] "RemoveContainer" containerID="2058ff56324d4814eba7505df61abbd33ef778a1c4853349732031ad9ba937f1" Aug 04 09:43:15.639323 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.639305 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2058ff56324d4814eba7505df61abbd33ef778a1c4853349732031ad9ba937f1"} err="failed to get container status \"2058ff56324d4814eba7505df61abbd33ef778a1c4853349732031ad9ba937f1\": rpc error: code = NotFound desc = could not find container \"2058ff56324d4814eba7505df61abbd33ef778a1c4853349732031ad9ba937f1\": container with ID starting with 2058ff56324d4814eba7505df61abbd33ef778a1c4853349732031ad9ba937f1 not found: ID does not exist" Aug 04 09:43:15.639364 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.639323 2572 scope.go:117] "RemoveContainer" containerID="8f23cd1f0e7b8e087b3de416f2284535a5b1e5236a0430b8f8982610a6066cab" Aug 04 09:43:15.639524 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.639507 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f23cd1f0e7b8e087b3de416f2284535a5b1e5236a0430b8f8982610a6066cab"} err="failed to get container status \"8f23cd1f0e7b8e087b3de416f2284535a5b1e5236a0430b8f8982610a6066cab\": rpc error: code = NotFound desc = could not find container \"8f23cd1f0e7b8e087b3de416f2284535a5b1e5236a0430b8f8982610a6066cab\": container with ID starting with 8f23cd1f0e7b8e087b3de416f2284535a5b1e5236a0430b8f8982610a6066cab not found: ID does not exist" Aug 04 09:43:15.639585 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.639524 2572 scope.go:117] "RemoveContainer" containerID="48be1d4a8fd45aba311286adc7583d434189c35f4e769e31fd109ee1b9f0634f" Aug 04 09:43:15.639752 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.639736 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48be1d4a8fd45aba311286adc7583d434189c35f4e769e31fd109ee1b9f0634f"} err="failed to get container status \"48be1d4a8fd45aba311286adc7583d434189c35f4e769e31fd109ee1b9f0634f\": rpc error: code = NotFound desc = could not find container \"48be1d4a8fd45aba311286adc7583d434189c35f4e769e31fd109ee1b9f0634f\": container with ID starting with 48be1d4a8fd45aba311286adc7583d434189c35f4e769e31fd109ee1b9f0634f not found: ID does not exist" Aug 04 09:43:15.653812 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.653768 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 04 09:43:15.654081 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.654068 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerName="kube-rbac-proxy" Aug 04 09:43:15.654124 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.654083 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerName="kube-rbac-proxy" Aug 04 09:43:15.654124 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.654094 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerName="kube-rbac-proxy-metric" Aug 04 09:43:15.654124 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.654099 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerName="kube-rbac-proxy-metric" Aug 04 09:43:15.654124 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.654112 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerName="kube-rbac-proxy-web" Aug 04 09:43:15.654124 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.654121 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerName="kube-rbac-proxy-web" Aug 04 09:43:15.654262 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.654128 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerName="init-config-reloader" Aug 04 09:43:15.654262 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.654136 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerName="init-config-reloader" Aug 04 09:43:15.654262 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.654150 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerName="config-reloader" Aug 04 09:43:15.654262 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.654155 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerName="config-reloader" Aug 04 09:43:15.654262 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.654162 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerName="prom-label-proxy" Aug 04 09:43:15.654262 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.654167 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerName="prom-label-proxy" Aug 04 09:43:15.654262 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.654172 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerName="alertmanager" Aug 04 09:43:15.654262 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.654177 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerName="alertmanager" Aug 04 09:43:15.654262 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.654226 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerName="config-reloader" Aug 04 09:43:15.654262 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.654234 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerName="kube-rbac-proxy-metric" Aug 04 09:43:15.654262 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.654241 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerName="prom-label-proxy" Aug 04 09:43:15.654262 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.654247 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerName="alertmanager" Aug 04 09:43:15.654262 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.654251 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerName="kube-rbac-proxy-web" Aug 04 09:43:15.654262 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.654256 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="59fb9607-ffdc-4604-82eb-46b1187f6628" containerName="kube-rbac-proxy" Aug 04 09:43:15.658093 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.658071 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.661923 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.661900 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Aug 04 09:43:15.662074 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.661994 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Aug 04 09:43:15.662074 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.662060 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Aug 04 09:43:15.662186 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.662069 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Aug 04 09:43:15.662233 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.662188 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Aug 04 09:43:15.662233 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.662204 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Aug 04 09:43:15.662320 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.662282 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Aug 04 09:43:15.662586 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.662571 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7v7ch\"" Aug 04 09:43:15.662586 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.662580 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Aug 04 09:43:15.668764 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.668743 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Aug 04 09:43:15.679322 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.679294 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 04 09:43:15.777518 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.777483 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59fb9607-ffdc-4604-82eb-46b1187f6628" path="/var/lib/kubelet/pods/59fb9607-ffdc-4604-82eb-46b1187f6628/volumes" Aug 04 09:43:15.806976 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.806944 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzr4z\" (UniqueName: \"kubernetes.io/projected/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-kube-api-access-xzr4z\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.806976 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.806982 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.807180 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.807011 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-config-volume\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.807180 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.807028 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.807180 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.807044 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.807180 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.807061 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.807180 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.807088 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.807180 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.807142 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-web-config\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.807180 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.807176 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.807390 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.807212 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-config-out\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.807390 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.807235 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.807390 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.807267 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.807390 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.807284 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.908203 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.908113 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzr4z\" (UniqueName: \"kubernetes.io/projected/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-kube-api-access-xzr4z\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.908203 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.908151 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.908203 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.908181 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-config-volume\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.908203 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.908204 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.908516 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.908232 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.908516 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.908257 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.908516 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.908303 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.908516 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.908336 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-web-config\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.908516 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.908360 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.908516 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.908404 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-config-out\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.908516 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.908436 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.908516 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.908470 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.908516 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.908499 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.908963 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.908946 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.909335 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.909308 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.909445 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.909352 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.911413 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.911380 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-config-volume\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.911520 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.911432 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.911716 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.911685 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.911848 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.911828 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.912067 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.912046 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-config-out\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.912131 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.912056 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.912171 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.912127 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.912296 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.912277 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.913266 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.913246 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-web-config\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.918427 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.918405 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzr4z\" (UniqueName: \"kubernetes.io/projected/ee3517f9-52ed-4ae1-aba4-5f3e3668b814-kube-api-access-xzr4z\") pod \"alertmanager-main-0\" (UID: \"ee3517f9-52ed-4ae1-aba4-5f3e3668b814\") " pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:15.968523 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:15.968486 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Aug 04 09:43:16.111526 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:16.111499 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 04 09:43:16.113272 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:43:16.113248 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee3517f9_52ed_4ae1_aba4_5f3e3668b814.slice/crio-35eaa42f5e6f91cfabb3dc9475a6ba6c346f50ebe01fbae9f63845f551852767 WatchSource:0}: Error finding container 35eaa42f5e6f91cfabb3dc9475a6ba6c346f50ebe01fbae9f63845f551852767: Status 404 returned error can't find the container with id 35eaa42f5e6f91cfabb3dc9475a6ba6c346f50ebe01fbae9f63845f551852767 Aug 04 09:43:16.589210 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:16.589167 2572 generic.go:356] "Generic (PLEG): container finished" podID="ee3517f9-52ed-4ae1-aba4-5f3e3668b814" containerID="742c0fd6e811a5a6b496dbf0460116fb094f9874451bad58c75deb2d19feab1d" exitCode=0 Aug 04 09:43:16.589504 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:16.589232 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee3517f9-52ed-4ae1-aba4-5f3e3668b814","Type":"ContainerDied","Data":"742c0fd6e811a5a6b496dbf0460116fb094f9874451bad58c75deb2d19feab1d"} Aug 04 09:43:16.589504 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:16.589311 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee3517f9-52ed-4ae1-aba4-5f3e3668b814","Type":"ContainerStarted","Data":"35eaa42f5e6f91cfabb3dc9475a6ba6c346f50ebe01fbae9f63845f551852767"} Aug 04 09:43:17.597557 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:17.597522 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee3517f9-52ed-4ae1-aba4-5f3e3668b814","Type":"ContainerStarted","Data":"8abb004fdd4d944dd62f7b77959d291a28f1917cbe88318d3b6dc48a3c949eba"} Aug 04 09:43:17.597965 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:17.597594 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee3517f9-52ed-4ae1-aba4-5f3e3668b814","Type":"ContainerStarted","Data":"40f80c6748f22b22357fae54c6daaccdd9176117da2464f2511f135390694ed0"} Aug 04 09:43:17.597965 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:17.597636 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee3517f9-52ed-4ae1-aba4-5f3e3668b814","Type":"ContainerStarted","Data":"34b813d2e3333df66f0258f41ee6ff4aca49ac69c24db909488fe113b17ddb6a"} Aug 04 09:43:17.597965 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:17.597665 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee3517f9-52ed-4ae1-aba4-5f3e3668b814","Type":"ContainerStarted","Data":"6913854e56922fddbff554cf9aee313ae2952d0bdabc22c9f4cc4b94bc0c0995"} Aug 04 09:43:17.597965 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:17.597687 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee3517f9-52ed-4ae1-aba4-5f3e3668b814","Type":"ContainerStarted","Data":"4762ea20197bda5c4754e964666603c6aca622805cafdda00060328f2f1976da"} Aug 04 09:43:17.597965 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:17.597723 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee3517f9-52ed-4ae1-aba4-5f3e3668b814","Type":"ContainerStarted","Data":"aaa9623c54889dc55939fad127f0e668a903eb1b13da144e7b2546690277a949"} Aug 04 09:43:17.633788 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:17.633723 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.617189401 podStartE2EDuration="2.633701277s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.511876ms" isStatefulPod=false podCreationTimestamp="2026-08-04 09:43:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:43:17.631146112 +0000 UTC m=+166.375280532" watchObservedRunningTime="2026-08-04 09:43:17.633701277 +0000 UTC m=+166.377835697" Aug 04 09:43:21.326228 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:21.326194 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:43:21.423094 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:21.423058 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:43:24.050408 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:24.050295 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-689b989dbb-xjpfk" Aug 04 09:43:24.050831 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:24.050442 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-689b989dbb-xjpfk" Aug 04 09:43:24.055280 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:24.055255 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-689b989dbb-xjpfk" Aug 04 09:43:24.632413 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:24.632382 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-689b989dbb-xjpfk" Aug 04 09:43:24.840132 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:24.840098 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:43:24.941040 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:24.940962 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:43:25.038640 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:25.038606 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:43:28.918575 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:28.918545 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:43:29.038738 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:43:29.038704 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:44:07.037670 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:07.037609 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:44:07.118298 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:07.118263 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:44:13.724645 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:13.724608 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:44:13.861385 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:13.861348 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:44:18.646758 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:18.646719 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:44:19.020087 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:19.020048 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:44:20.596119 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:20.596087 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t"] Aug 04 09:44:20.598676 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:20.598659 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t" Aug 04 09:44:20.604655 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:20.604620 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Aug 04 09:44:20.604852 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:20.604667 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Aug 04 09:44:20.605734 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:20.605714 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-grkw6\"" Aug 04 09:44:20.625199 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:20.625161 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t"] Aug 04 09:44:20.755233 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:20.755190 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2307bdd-dcbb-4f70-b9e3-1969b1425887-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t\" (UID: \"d2307bdd-dcbb-4f70-b9e3-1969b1425887\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t" Aug 04 09:44:20.755453 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:20.755246 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlk7c\" (UniqueName: \"kubernetes.io/projected/d2307bdd-dcbb-4f70-b9e3-1969b1425887-kube-api-access-nlk7c\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t\" (UID: \"d2307bdd-dcbb-4f70-b9e3-1969b1425887\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t" Aug 04 09:44:20.755453 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:20.755334 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2307bdd-dcbb-4f70-b9e3-1969b1425887-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t\" (UID: \"d2307bdd-dcbb-4f70-b9e3-1969b1425887\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t" Aug 04 09:44:20.856441 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:20.856347 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2307bdd-dcbb-4f70-b9e3-1969b1425887-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t\" (UID: \"d2307bdd-dcbb-4f70-b9e3-1969b1425887\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t" Aug 04 09:44:20.856441 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:20.856408 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlk7c\" (UniqueName: \"kubernetes.io/projected/d2307bdd-dcbb-4f70-b9e3-1969b1425887-kube-api-access-nlk7c\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t\" (UID: \"d2307bdd-dcbb-4f70-b9e3-1969b1425887\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t" Aug 04 09:44:20.856618 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:20.856466 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2307bdd-dcbb-4f70-b9e3-1969b1425887-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t\" (UID: \"d2307bdd-dcbb-4f70-b9e3-1969b1425887\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t" Aug 04 09:44:20.856852 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:20.856832 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2307bdd-dcbb-4f70-b9e3-1969b1425887-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t\" (UID: \"d2307bdd-dcbb-4f70-b9e3-1969b1425887\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t" Aug 04 09:44:20.856895 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:20.856842 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2307bdd-dcbb-4f70-b9e3-1969b1425887-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t\" (UID: \"d2307bdd-dcbb-4f70-b9e3-1969b1425887\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t" Aug 04 09:44:20.868913 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:20.868876 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlk7c\" (UniqueName: \"kubernetes.io/projected/d2307bdd-dcbb-4f70-b9e3-1969b1425887-kube-api-access-nlk7c\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t\" (UID: \"d2307bdd-dcbb-4f70-b9e3-1969b1425887\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t" Aug 04 09:44:20.907494 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:20.907453 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t" Aug 04 09:44:21.052539 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:21.052513 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t"] Aug 04 09:44:21.054673 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:44:21.054641 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2307bdd_dcbb_4f70_b9e3_1969b1425887.slice/crio-e3012085b61d875395907be838793d0b7c628fa52650a32fa681e17dd2576599 WatchSource:0}: Error finding container e3012085b61d875395907be838793d0b7c628fa52650a32fa681e17dd2576599: Status 404 returned error can't find the container with id e3012085b61d875395907be838793d0b7c628fa52650a32fa681e17dd2576599 Aug 04 09:44:21.838845 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:21.838792 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t" event={"ID":"d2307bdd-dcbb-4f70-b9e3-1969b1425887","Type":"ContainerStarted","Data":"e3012085b61d875395907be838793d0b7c628fa52650a32fa681e17dd2576599"} Aug 04 09:44:26.863404 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:26.863364 2572 generic.go:356] "Generic (PLEG): container finished" podID="d2307bdd-dcbb-4f70-b9e3-1969b1425887" containerID="0099921c2fe1c1aa19597f17f7f17dbede124c36ab58d8430730c3b35eadb852" exitCode=0 Aug 04 09:44:26.863898 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:26.863422 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t" event={"ID":"d2307bdd-dcbb-4f70-b9e3-1969b1425887","Type":"ContainerDied","Data":"0099921c2fe1c1aa19597f17f7f17dbede124c36ab58d8430730c3b35eadb852"} Aug 04 09:44:29.877875 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:29.877835 2572 generic.go:356] "Generic (PLEG): container finished" podID="d2307bdd-dcbb-4f70-b9e3-1969b1425887" containerID="737a8279bda99dc7ee1c8689c3385ded4a834271ba465f64201d9a734736b5f0" exitCode=0 Aug 04 09:44:29.878329 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:29.877920 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t" event={"ID":"d2307bdd-dcbb-4f70-b9e3-1969b1425887","Type":"ContainerDied","Data":"737a8279bda99dc7ee1c8689c3385ded4a834271ba465f64201d9a734736b5f0"} Aug 04 09:44:36.908467 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:36.908426 2572 generic.go:356] "Generic (PLEG): container finished" podID="d2307bdd-dcbb-4f70-b9e3-1969b1425887" containerID="9d0036bee5bb626e2e3d2faa209a28fba67a84199017924f40e308fd8d806811" exitCode=0 Aug 04 09:44:36.908934 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:36.908505 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t" event={"ID":"d2307bdd-dcbb-4f70-b9e3-1969b1425887","Type":"ContainerDied","Data":"9d0036bee5bb626e2e3d2faa209a28fba67a84199017924f40e308fd8d806811"} Aug 04 09:44:38.040950 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:38.040923 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t" Aug 04 09:44:38.110086 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:38.110051 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2307bdd-dcbb-4f70-b9e3-1969b1425887-util\") pod \"d2307bdd-dcbb-4f70-b9e3-1969b1425887\" (UID: \"d2307bdd-dcbb-4f70-b9e3-1969b1425887\") " Aug 04 09:44:38.110086 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:38.110096 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2307bdd-dcbb-4f70-b9e3-1969b1425887-bundle\") pod \"d2307bdd-dcbb-4f70-b9e3-1969b1425887\" (UID: \"d2307bdd-dcbb-4f70-b9e3-1969b1425887\") " Aug 04 09:44:38.110302 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:38.110121 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nlk7c\" (UniqueName: \"kubernetes.io/projected/d2307bdd-dcbb-4f70-b9e3-1969b1425887-kube-api-access-nlk7c\") pod \"d2307bdd-dcbb-4f70-b9e3-1969b1425887\" (UID: \"d2307bdd-dcbb-4f70-b9e3-1969b1425887\") " Aug 04 09:44:38.110742 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:38.110706 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2307bdd-dcbb-4f70-b9e3-1969b1425887-bundle" (OuterVolumeSpecName: "bundle") pod "d2307bdd-dcbb-4f70-b9e3-1969b1425887" (UID: "d2307bdd-dcbb-4f70-b9e3-1969b1425887"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:44:38.112373 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:38.112342 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2307bdd-dcbb-4f70-b9e3-1969b1425887-kube-api-access-nlk7c" (OuterVolumeSpecName: "kube-api-access-nlk7c") pod "d2307bdd-dcbb-4f70-b9e3-1969b1425887" (UID: "d2307bdd-dcbb-4f70-b9e3-1969b1425887"). InnerVolumeSpecName "kube-api-access-nlk7c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:44:38.114235 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:38.114203 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2307bdd-dcbb-4f70-b9e3-1969b1425887-util" (OuterVolumeSpecName: "util") pod "d2307bdd-dcbb-4f70-b9e3-1969b1425887" (UID: "d2307bdd-dcbb-4f70-b9e3-1969b1425887"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:44:38.211502 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:38.211469 2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2307bdd-dcbb-4f70-b9e3-1969b1425887-util\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:44:38.211502 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:38.211500 2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2307bdd-dcbb-4f70-b9e3-1969b1425887-bundle\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:44:38.211691 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:38.211510 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nlk7c\" (UniqueName: \"kubernetes.io/projected/d2307bdd-dcbb-4f70-b9e3-1969b1425887-kube-api-access-nlk7c\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:44:38.918010 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:38.917973 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t" event={"ID":"d2307bdd-dcbb-4f70-b9e3-1969b1425887","Type":"ContainerDied","Data":"e3012085b61d875395907be838793d0b7c628fa52650a32fa681e17dd2576599"} Aug 04 09:44:38.918010 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:38.918014 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404lf4t" Aug 04 09:44:38.918206 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:38.918023 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e3012085b61d875395907be838793d0b7c628fa52650a32fa681e17dd2576599" Aug 04 09:44:42.308334 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:42.308301 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:44:42.487191 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:42.487160 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:44:48.632503 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:48.632467 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:44:48.802098 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:48.802055 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:44:51.259505 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:51.259470 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:44:51.487141 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:51.487106 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:44:52.161458 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:52.161420 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:44:52.431213 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:52.431126 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:44:52.608836 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:52.608781 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t"] Aug 04 09:44:52.609211 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:52.609193 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2307bdd-dcbb-4f70-b9e3-1969b1425887" containerName="extract" Aug 04 09:44:52.609292 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:52.609214 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2307bdd-dcbb-4f70-b9e3-1969b1425887" containerName="extract" Aug 04 09:44:52.609292 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:52.609231 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2307bdd-dcbb-4f70-b9e3-1969b1425887" containerName="util" Aug 04 09:44:52.609292 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:52.609239 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2307bdd-dcbb-4f70-b9e3-1969b1425887" containerName="util" Aug 04 09:44:52.609292 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:52.609261 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2307bdd-dcbb-4f70-b9e3-1969b1425887" containerName="pull" Aug 04 09:44:52.609292 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:52.609271 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2307bdd-dcbb-4f70-b9e3-1969b1425887" containerName="pull" Aug 04 09:44:52.609541 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:52.609339 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2307bdd-dcbb-4f70-b9e3-1969b1425887" containerName="extract" Aug 04 09:44:52.612951 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:52.612923 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t" Aug 04 09:44:52.616382 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:52.616359 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-grkw6\"" Aug 04 09:44:52.616382 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:52.616361 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Aug 04 09:44:52.617495 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:52.617480 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Aug 04 09:44:52.629316 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:52.629282 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t"] Aug 04 09:44:52.730824 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:52.730759 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d2d3c3f-a754-4ac4-bf21-75e075f103b9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t\" (UID: \"9d2d3c3f-a754-4ac4-bf21-75e075f103b9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t" Aug 04 09:44:52.731009 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:52.730839 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvggr\" (UniqueName: \"kubernetes.io/projected/9d2d3c3f-a754-4ac4-bf21-75e075f103b9-kube-api-access-wvggr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t\" (UID: \"9d2d3c3f-a754-4ac4-bf21-75e075f103b9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t" Aug 04 09:44:52.731009 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:52.730896 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d2d3c3f-a754-4ac4-bf21-75e075f103b9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t\" (UID: \"9d2d3c3f-a754-4ac4-bf21-75e075f103b9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t" Aug 04 09:44:52.832470 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:52.832433 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d2d3c3f-a754-4ac4-bf21-75e075f103b9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t\" (UID: \"9d2d3c3f-a754-4ac4-bf21-75e075f103b9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t" Aug 04 09:44:52.832775 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:52.832744 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d2d3c3f-a754-4ac4-bf21-75e075f103b9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t\" (UID: \"9d2d3c3f-a754-4ac4-bf21-75e075f103b9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t" Aug 04 09:44:52.832958 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:52.832886 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d2d3c3f-a754-4ac4-bf21-75e075f103b9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t\" (UID: \"9d2d3c3f-a754-4ac4-bf21-75e075f103b9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t" Aug 04 09:44:52.838054 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:52.833321 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvggr\" (UniqueName: \"kubernetes.io/projected/9d2d3c3f-a754-4ac4-bf21-75e075f103b9-kube-api-access-wvggr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t\" (UID: \"9d2d3c3f-a754-4ac4-bf21-75e075f103b9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t" Aug 04 09:44:52.838054 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:52.833480 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d2d3c3f-a754-4ac4-bf21-75e075f103b9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t\" (UID: \"9d2d3c3f-a754-4ac4-bf21-75e075f103b9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t" Aug 04 09:44:52.862257 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:52.862218 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvggr\" (UniqueName: \"kubernetes.io/projected/9d2d3c3f-a754-4ac4-bf21-75e075f103b9-kube-api-access-wvggr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t\" (UID: \"9d2d3c3f-a754-4ac4-bf21-75e075f103b9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t" Aug 04 09:44:52.922247 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:52.922202 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t" Aug 04 09:44:53.104310 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:53.104274 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t"] Aug 04 09:44:53.107914 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:44:53.107884 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d2d3c3f_a754_4ac4_bf21_75e075f103b9.slice/crio-72fcc8baad2b0da36a5c50b25d904694a16c1abf55d9946c45b5b5e4fa6c3f89 WatchSource:0}: Error finding container 72fcc8baad2b0da36a5c50b25d904694a16c1abf55d9946c45b5b5e4fa6c3f89: Status 404 returned error can't find the container with id 72fcc8baad2b0da36a5c50b25d904694a16c1abf55d9946c45b5b5e4fa6c3f89 Aug 04 09:44:53.773536 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:53.773497 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:44:53.974937 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:53.974902 2572 generic.go:356] "Generic (PLEG): container finished" podID="9d2d3c3f-a754-4ac4-bf21-75e075f103b9" containerID="e760979b38a525418b8336f544cf42907ecdfe57e5db640bedf18ac31ddec9d4" exitCode=0 Aug 04 09:44:53.975131 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:53.975002 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t" event={"ID":"9d2d3c3f-a754-4ac4-bf21-75e075f103b9","Type":"ContainerDied","Data":"e760979b38a525418b8336f544cf42907ecdfe57e5db640bedf18ac31ddec9d4"} Aug 04 09:44:53.975131 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:53.975063 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t" event={"ID":"9d2d3c3f-a754-4ac4-bf21-75e075f103b9","Type":"ContainerStarted","Data":"72fcc8baad2b0da36a5c50b25d904694a16c1abf55d9946c45b5b5e4fa6c3f89"} Aug 04 09:44:54.056620 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:54.056531 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:44:55.415266 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:55.415226 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:44:55.543067 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:55.543033 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:44:56.989532 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:56.989496 2572 generic.go:356] "Generic (PLEG): container finished" podID="9d2d3c3f-a754-4ac4-bf21-75e075f103b9" containerID="f513d80be5052652ae53181a3b332a917f28915b56e35f09fc0502ba3fc95b68" exitCode=0 Aug 04 09:44:56.989957 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:56.989578 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t" event={"ID":"9d2d3c3f-a754-4ac4-bf21-75e075f103b9","Type":"ContainerDied","Data":"f513d80be5052652ae53181a3b332a917f28915b56e35f09fc0502ba3fc95b68"} Aug 04 09:44:57.995572 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:57.995539 2572 generic.go:356] "Generic (PLEG): container finished" podID="9d2d3c3f-a754-4ac4-bf21-75e075f103b9" containerID="999f4367687ba06844f9d580fb8ef3805f92003f4a5e55bb06dd723d9bf6414d" exitCode=0 Aug 04 09:44:57.995994 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:57.995624 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t" event={"ID":"9d2d3c3f-a754-4ac4-bf21-75e075f103b9","Type":"ContainerDied","Data":"999f4367687ba06844f9d580fb8ef3805f92003f4a5e55bb06dd723d9bf6414d"} Aug 04 09:44:58.572404 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:58.572370 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-4lp97"] Aug 04 09:44:58.574644 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:58.574626 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-4lp97" Aug 04 09:44:58.581923 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:58.581887 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Aug 04 09:44:58.584990 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:58.584972 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-2vhtk\"" Aug 04 09:44:58.591351 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:58.591320 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Aug 04 09:44:58.593184 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:58.593162 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-4lp97"] Aug 04 09:44:58.689134 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:58.689095 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02bb1346-62e9-4d5d-8da5-9631a0a6d019-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-4lp97\" (UID: \"02bb1346-62e9-4d5d-8da5-9631a0a6d019\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4lp97" Aug 04 09:44:58.689318 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:58.689149 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpf26\" (UniqueName: \"kubernetes.io/projected/02bb1346-62e9-4d5d-8da5-9631a0a6d019-kube-api-access-rpf26\") pod \"cert-manager-webhook-6486ffd997-4lp97\" (UID: \"02bb1346-62e9-4d5d-8da5-9631a0a6d019\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4lp97" Aug 04 09:44:58.791002 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:58.790955 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpf26\" (UniqueName: \"kubernetes.io/projected/02bb1346-62e9-4d5d-8da5-9631a0a6d019-kube-api-access-rpf26\") pod \"cert-manager-webhook-6486ffd997-4lp97\" (UID: \"02bb1346-62e9-4d5d-8da5-9631a0a6d019\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4lp97" Aug 04 09:44:58.791194 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:58.791137 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02bb1346-62e9-4d5d-8da5-9631a0a6d019-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-4lp97\" (UID: \"02bb1346-62e9-4d5d-8da5-9631a0a6d019\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4lp97" Aug 04 09:44:58.803961 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:58.803927 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02bb1346-62e9-4d5d-8da5-9631a0a6d019-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-4lp97\" (UID: \"02bb1346-62e9-4d5d-8da5-9631a0a6d019\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4lp97" Aug 04 09:44:58.805215 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:58.805191 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpf26\" (UniqueName: \"kubernetes.io/projected/02bb1346-62e9-4d5d-8da5-9631a0a6d019-kube-api-access-rpf26\") pod \"cert-manager-webhook-6486ffd997-4lp97\" (UID: \"02bb1346-62e9-4d5d-8da5-9631a0a6d019\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4lp97" Aug 04 09:44:58.883348 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:58.883255 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-4lp97" Aug 04 09:44:59.033561 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:59.033526 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-4lp97"] Aug 04 09:44:59.034650 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:44:59.034625 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02bb1346_62e9_4d5d_8da5_9631a0a6d019.slice/crio-70e6fdfe3c9c9199e688a1b0b502019af627e212ccd5eabfa02a06d73765aa1a WatchSource:0}: Error finding container 70e6fdfe3c9c9199e688a1b0b502019af627e212ccd5eabfa02a06d73765aa1a: Status 404 returned error can't find the container with id 70e6fdfe3c9c9199e688a1b0b502019af627e212ccd5eabfa02a06d73765aa1a Aug 04 09:44:59.140609 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:59.140542 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t" Aug 04 09:44:59.195083 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:59.195042 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d2d3c3f-a754-4ac4-bf21-75e075f103b9-util\") pod \"9d2d3c3f-a754-4ac4-bf21-75e075f103b9\" (UID: \"9d2d3c3f-a754-4ac4-bf21-75e075f103b9\") " Aug 04 09:44:59.195083 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:59.195086 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d2d3c3f-a754-4ac4-bf21-75e075f103b9-bundle\") pod \"9d2d3c3f-a754-4ac4-bf21-75e075f103b9\" (UID: \"9d2d3c3f-a754-4ac4-bf21-75e075f103b9\") " Aug 04 09:44:59.195338 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:59.195125 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvggr\" (UniqueName: \"kubernetes.io/projected/9d2d3c3f-a754-4ac4-bf21-75e075f103b9-kube-api-access-wvggr\") pod \"9d2d3c3f-a754-4ac4-bf21-75e075f103b9\" (UID: \"9d2d3c3f-a754-4ac4-bf21-75e075f103b9\") " Aug 04 09:44:59.195531 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:59.195502 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d2d3c3f-a754-4ac4-bf21-75e075f103b9-bundle" (OuterVolumeSpecName: "bundle") pod "9d2d3c3f-a754-4ac4-bf21-75e075f103b9" (UID: "9d2d3c3f-a754-4ac4-bf21-75e075f103b9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:44:59.197262 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:59.197238 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d2d3c3f-a754-4ac4-bf21-75e075f103b9-kube-api-access-wvggr" (OuterVolumeSpecName: "kube-api-access-wvggr") pod "9d2d3c3f-a754-4ac4-bf21-75e075f103b9" (UID: "9d2d3c3f-a754-4ac4-bf21-75e075f103b9"). InnerVolumeSpecName "kube-api-access-wvggr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:44:59.200071 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:59.200044 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d2d3c3f-a754-4ac4-bf21-75e075f103b9-util" (OuterVolumeSpecName: "util") pod "9d2d3c3f-a754-4ac4-bf21-75e075f103b9" (UID: "9d2d3c3f-a754-4ac4-bf21-75e075f103b9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:44:59.296225 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:59.296186 2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d2d3c3f-a754-4ac4-bf21-75e075f103b9-util\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:44:59.296225 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:59.296217 2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d2d3c3f-a754-4ac4-bf21-75e075f103b9-bundle\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:44:59.296225 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:44:59.296229 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvggr\" (UniqueName: \"kubernetes.io/projected/9d2d3c3f-a754-4ac4-bf21-75e075f103b9-kube-api-access-wvggr\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:45:00.007220 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:00.007165 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t" event={"ID":"9d2d3c3f-a754-4ac4-bf21-75e075f103b9","Type":"ContainerDied","Data":"72fcc8baad2b0da36a5c50b25d904694a16c1abf55d9946c45b5b5e4fa6c3f89"} Aug 04 09:45:00.007408 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:00.007230 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="72fcc8baad2b0da36a5c50b25d904694a16c1abf55d9946c45b5b5e4fa6c3f89" Aug 04 09:45:00.007408 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:00.007192 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbfw8t" Aug 04 09:45:00.008608 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:00.008565 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-4lp97" event={"ID":"02bb1346-62e9-4d5d-8da5-9631a0a6d019","Type":"ContainerStarted","Data":"70e6fdfe3c9c9199e688a1b0b502019af627e212ccd5eabfa02a06d73765aa1a"} Aug 04 09:45:01.180915 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:01.180864 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:01.729697 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:01.729610 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:02.019076 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:02.018980 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-4lp97" event={"ID":"02bb1346-62e9-4d5d-8da5-9631a0a6d019","Type":"ContainerStarted","Data":"0707184a3a9690797572f43fef8c66c4dd2c1f262174baea0389bbaef3d4fe84"} Aug 04 09:45:02.019235 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:02.019098 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-4lp97" Aug 04 09:45:02.039782 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:02.039724 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-4lp97" podStartSLOduration=1.610813946 podStartE2EDuration="4.039707114s" totalImagesPullingTime="2.428893168s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:45:02.038634947 +0000 UTC m=+270.782769372" watchObservedRunningTime="2026-08-04 09:45:02.039707114 +0000 UTC m=+270.783841532" Aug 04 09:45:02.107224 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:02.107189 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:02.206335 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:02.206302 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:02.911200 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:02.911171 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:03.830830 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:03.830782 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:04.115817 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:04.115712 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:08.026130 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:08.026098 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-4lp97" Aug 04 09:45:09.377106 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:09.377070 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:09.476433 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:09.476401 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:11.475694 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:11.475656 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp"] Aug 04 09:45:11.476101 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:11.476009 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d2d3c3f-a754-4ac4-bf21-75e075f103b9" containerName="util" Aug 04 09:45:11.476101 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:11.476021 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d2d3c3f-a754-4ac4-bf21-75e075f103b9" containerName="util" Aug 04 09:45:11.476101 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:11.476031 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d2d3c3f-a754-4ac4-bf21-75e075f103b9" containerName="pull" Aug 04 09:45:11.476101 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:11.476036 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d2d3c3f-a754-4ac4-bf21-75e075f103b9" containerName="pull" Aug 04 09:45:11.476101 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:11.476043 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d2d3c3f-a754-4ac4-bf21-75e075f103b9" containerName="extract" Aug 04 09:45:11.476101 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:11.476048 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d2d3c3f-a754-4ac4-bf21-75e075f103b9" containerName="extract" Aug 04 09:45:11.476296 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:11.476110 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d2d3c3f-a754-4ac4-bf21-75e075f103b9" containerName="extract" Aug 04 09:45:11.478217 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:11.478195 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp" Aug 04 09:45:11.481911 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:11.481880 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Aug 04 09:45:11.482042 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:11.481880 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Aug 04 09:45:11.482042 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:11.481918 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-grkw6\"" Aug 04 09:45:11.491484 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:11.491456 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp"] Aug 04 09:45:11.609074 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:11.609027 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzcmq\" (UniqueName: \"kubernetes.io/projected/461fe16e-e0d8-4f41-b07a-35a53901814c-kube-api-access-qzcmq\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp\" (UID: \"461fe16e-e0d8-4f41-b07a-35a53901814c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp" Aug 04 09:45:11.609255 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:11.609111 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/461fe16e-e0d8-4f41-b07a-35a53901814c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp\" (UID: \"461fe16e-e0d8-4f41-b07a-35a53901814c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp" Aug 04 09:45:11.609255 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:11.609144 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/461fe16e-e0d8-4f41-b07a-35a53901814c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp\" (UID: \"461fe16e-e0d8-4f41-b07a-35a53901814c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp" Aug 04 09:45:11.629254 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:11.629223 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:11.709558 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:11.709517 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/461fe16e-e0d8-4f41-b07a-35a53901814c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp\" (UID: \"461fe16e-e0d8-4f41-b07a-35a53901814c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp" Aug 04 09:45:11.709758 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:11.709570 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/461fe16e-e0d8-4f41-b07a-35a53901814c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp\" (UID: \"461fe16e-e0d8-4f41-b07a-35a53901814c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp" Aug 04 09:45:11.709758 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:11.709643 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzcmq\" (UniqueName: \"kubernetes.io/projected/461fe16e-e0d8-4f41-b07a-35a53901814c-kube-api-access-qzcmq\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp\" (UID: \"461fe16e-e0d8-4f41-b07a-35a53901814c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp" Aug 04 09:45:11.709989 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:11.709967 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/461fe16e-e0d8-4f41-b07a-35a53901814c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp\" (UID: \"461fe16e-e0d8-4f41-b07a-35a53901814c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp" Aug 04 09:45:11.710055 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:11.709986 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/461fe16e-e0d8-4f41-b07a-35a53901814c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp\" (UID: \"461fe16e-e0d8-4f41-b07a-35a53901814c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp" Aug 04 09:45:11.719564 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:11.719529 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzcmq\" (UniqueName: \"kubernetes.io/projected/461fe16e-e0d8-4f41-b07a-35a53901814c-kube-api-access-qzcmq\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp\" (UID: \"461fe16e-e0d8-4f41-b07a-35a53901814c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp" Aug 04 09:45:11.787390 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:11.787288 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp" Aug 04 09:45:11.927132 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:11.927105 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp"] Aug 04 09:45:11.928890 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:45:11.928853 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod461fe16e_e0d8_4f41_b07a_35a53901814c.slice/crio-1681848c1de31cba757f650d8a0844900263521b5bd6bb8799647c02ec5265af WatchSource:0}: Error finding container 1681848c1de31cba757f650d8a0844900263521b5bd6bb8799647c02ec5265af: Status 404 returned error can't find the container with id 1681848c1de31cba757f650d8a0844900263521b5bd6bb8799647c02ec5265af Aug 04 09:45:12.058340 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:12.058253 2572 generic.go:356] "Generic (PLEG): container finished" podID="461fe16e-e0d8-4f41-b07a-35a53901814c" containerID="f9499d7cda486e023396b98fe9d27277b5064be0621e8e8f0e4a336eb634e0e0" exitCode=0 Aug 04 09:45:12.058484 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:12.058336 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp" event={"ID":"461fe16e-e0d8-4f41-b07a-35a53901814c","Type":"ContainerDied","Data":"f9499d7cda486e023396b98fe9d27277b5064be0621e8e8f0e4a336eb634e0e0"} Aug 04 09:45:12.058484 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:12.058391 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp" event={"ID":"461fe16e-e0d8-4f41-b07a-35a53901814c","Type":"ContainerStarted","Data":"1681848c1de31cba757f650d8a0844900263521b5bd6bb8799647c02ec5265af"} Aug 04 09:45:14.068091 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:14.068050 2572 generic.go:356] "Generic (PLEG): container finished" podID="461fe16e-e0d8-4f41-b07a-35a53901814c" containerID="3f734a1cbfd74b162f0e617099da90faf3c03841dee0a3745ecb25235582005c" exitCode=0 Aug 04 09:45:14.068583 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:14.068099 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp" event={"ID":"461fe16e-e0d8-4f41-b07a-35a53901814c","Type":"ContainerDied","Data":"3f734a1cbfd74b162f0e617099da90faf3c03841dee0a3745ecb25235582005c"} Aug 04 09:45:15.074205 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:15.074171 2572 generic.go:356] "Generic (PLEG): container finished" podID="461fe16e-e0d8-4f41-b07a-35a53901814c" containerID="56d6a4cf7ec9c8b01471912fa535c338b33465d9be073d7d7218591c76c3a9d6" exitCode=0 Aug 04 09:45:15.074588 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:15.074279 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp" event={"ID":"461fe16e-e0d8-4f41-b07a-35a53901814c","Type":"ContainerDied","Data":"56d6a4cf7ec9c8b01471912fa535c338b33465d9be073d7d7218591c76c3a9d6"} Aug 04 09:45:16.209946 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:16.209912 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp" Aug 04 09:45:16.350566 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:16.350473 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/461fe16e-e0d8-4f41-b07a-35a53901814c-bundle\") pod \"461fe16e-e0d8-4f41-b07a-35a53901814c\" (UID: \"461fe16e-e0d8-4f41-b07a-35a53901814c\") " Aug 04 09:45:16.350566 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:16.350517 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qzcmq\" (UniqueName: \"kubernetes.io/projected/461fe16e-e0d8-4f41-b07a-35a53901814c-kube-api-access-qzcmq\") pod \"461fe16e-e0d8-4f41-b07a-35a53901814c\" (UID: \"461fe16e-e0d8-4f41-b07a-35a53901814c\") " Aug 04 09:45:16.350566 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:16.350549 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/461fe16e-e0d8-4f41-b07a-35a53901814c-util\") pod \"461fe16e-e0d8-4f41-b07a-35a53901814c\" (UID: \"461fe16e-e0d8-4f41-b07a-35a53901814c\") " Aug 04 09:45:16.351217 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:16.351192 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/461fe16e-e0d8-4f41-b07a-35a53901814c-bundle" (OuterVolumeSpecName: "bundle") pod "461fe16e-e0d8-4f41-b07a-35a53901814c" (UID: "461fe16e-e0d8-4f41-b07a-35a53901814c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:45:16.352605 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:16.352579 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/461fe16e-e0d8-4f41-b07a-35a53901814c-kube-api-access-qzcmq" (OuterVolumeSpecName: "kube-api-access-qzcmq") pod "461fe16e-e0d8-4f41-b07a-35a53901814c" (UID: "461fe16e-e0d8-4f41-b07a-35a53901814c"). InnerVolumeSpecName "kube-api-access-qzcmq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:45:16.355959 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:16.355935 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/461fe16e-e0d8-4f41-b07a-35a53901814c-util" (OuterVolumeSpecName: "util") pod "461fe16e-e0d8-4f41-b07a-35a53901814c" (UID: "461fe16e-e0d8-4f41-b07a-35a53901814c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:45:16.451145 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:16.451104 2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/461fe16e-e0d8-4f41-b07a-35a53901814c-bundle\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:45:16.451145 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:16.451138 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzcmq\" (UniqueName: \"kubernetes.io/projected/461fe16e-e0d8-4f41-b07a-35a53901814c-kube-api-access-qzcmq\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:45:16.451145 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:16.451149 2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/461fe16e-e0d8-4f41-b07a-35a53901814c-util\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:45:17.084472 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:17.084441 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp" Aug 04 09:45:17.084641 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:17.084469 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnprp" event={"ID":"461fe16e-e0d8-4f41-b07a-35a53901814c","Type":"ContainerDied","Data":"1681848c1de31cba757f650d8a0844900263521b5bd6bb8799647c02ec5265af"} Aug 04 09:45:17.084641 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:17.084515 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1681848c1de31cba757f650d8a0844900263521b5bd6bb8799647c02ec5265af" Aug 04 09:45:23.221512 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:23.221473 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:23.340119 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:23.340081 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:23.516994 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:23.516903 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:23.725613 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:23.725581 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:24.066052 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:24.066022 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:25.107606 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.107567 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:25.224436 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.224399 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:25.243273 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.243231 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bb649b865-shttb"] Aug 04 09:45:25.243620 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.243603 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="461fe16e-e0d8-4f41-b07a-35a53901814c" containerName="pull" Aug 04 09:45:25.243703 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.243622 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="461fe16e-e0d8-4f41-b07a-35a53901814c" containerName="pull" Aug 04 09:45:25.243703 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.243634 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="461fe16e-e0d8-4f41-b07a-35a53901814c" containerName="extract" Aug 04 09:45:25.243703 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.243641 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="461fe16e-e0d8-4f41-b07a-35a53901814c" containerName="extract" Aug 04 09:45:25.243703 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.243654 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="461fe16e-e0d8-4f41-b07a-35a53901814c" containerName="util" Aug 04 09:45:25.243703 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.243664 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="461fe16e-e0d8-4f41-b07a-35a53901814c" containerName="util" Aug 04 09:45:25.243977 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.243768 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="461fe16e-e0d8-4f41-b07a-35a53901814c" containerName="extract" Aug 04 09:45:25.246605 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.246582 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-shttb" Aug 04 09:45:25.252567 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.252539 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Aug 04 09:45:25.252752 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.252722 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-9nfsq\"" Aug 04 09:45:25.253247 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.253105 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Aug 04 09:45:25.253465 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.253447 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Aug 04 09:45:25.255826 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.255791 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Aug 04 09:45:25.281225 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.281190 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bb649b865-shttb"] Aug 04 09:45:25.324152 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.324116 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a6865d23-1fea-4c03-87d9-b6cba1b0cebe-manifests\") pod \"opendatahub-operator-controller-manager-7bb649b865-shttb\" (UID: \"a6865d23-1fea-4c03-87d9-b6cba1b0cebe\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-shttb" Aug 04 09:45:25.324337 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.324177 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a6865d23-1fea-4c03-87d9-b6cba1b0cebe-tmp\") pod \"opendatahub-operator-controller-manager-7bb649b865-shttb\" (UID: \"a6865d23-1fea-4c03-87d9-b6cba1b0cebe\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-shttb" Aug 04 09:45:25.324337 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.324198 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a6865d23-1fea-4c03-87d9-b6cba1b0cebe-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bb649b865-shttb\" (UID: \"a6865d23-1fea-4c03-87d9-b6cba1b0cebe\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-shttb" Aug 04 09:45:25.324337 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.324230 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a6865d23-1fea-4c03-87d9-b6cba1b0cebe-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bb649b865-shttb\" (UID: \"a6865d23-1fea-4c03-87d9-b6cba1b0cebe\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-shttb" Aug 04 09:45:25.324337 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.324250 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcxwl\" (UniqueName: \"kubernetes.io/projected/a6865d23-1fea-4c03-87d9-b6cba1b0cebe-kube-api-access-wcxwl\") pod \"opendatahub-operator-controller-manager-7bb649b865-shttb\" (UID: \"a6865d23-1fea-4c03-87d9-b6cba1b0cebe\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-shttb" Aug 04 09:45:25.425109 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.425004 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a6865d23-1fea-4c03-87d9-b6cba1b0cebe-tmp\") pod \"opendatahub-operator-controller-manager-7bb649b865-shttb\" (UID: \"a6865d23-1fea-4c03-87d9-b6cba1b0cebe\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-shttb" Aug 04 09:45:25.425109 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.425051 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a6865d23-1fea-4c03-87d9-b6cba1b0cebe-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bb649b865-shttb\" (UID: \"a6865d23-1fea-4c03-87d9-b6cba1b0cebe\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-shttb" Aug 04 09:45:25.425109 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.425080 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a6865d23-1fea-4c03-87d9-b6cba1b0cebe-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bb649b865-shttb\" (UID: \"a6865d23-1fea-4c03-87d9-b6cba1b0cebe\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-shttb" Aug 04 09:45:25.425412 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.425112 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcxwl\" (UniqueName: \"kubernetes.io/projected/a6865d23-1fea-4c03-87d9-b6cba1b0cebe-kube-api-access-wcxwl\") pod \"opendatahub-operator-controller-manager-7bb649b865-shttb\" (UID: \"a6865d23-1fea-4c03-87d9-b6cba1b0cebe\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-shttb" Aug 04 09:45:25.425412 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.425175 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a6865d23-1fea-4c03-87d9-b6cba1b0cebe-manifests\") pod \"opendatahub-operator-controller-manager-7bb649b865-shttb\" (UID: \"a6865d23-1fea-4c03-87d9-b6cba1b0cebe\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-shttb" Aug 04 09:45:25.425602 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.425577 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a6865d23-1fea-4c03-87d9-b6cba1b0cebe-manifests\") pod \"opendatahub-operator-controller-manager-7bb649b865-shttb\" (UID: \"a6865d23-1fea-4c03-87d9-b6cba1b0cebe\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-shttb" Aug 04 09:45:25.427292 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.427264 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a6865d23-1fea-4c03-87d9-b6cba1b0cebe-tmp\") pod \"opendatahub-operator-controller-manager-7bb649b865-shttb\" (UID: \"a6865d23-1fea-4c03-87d9-b6cba1b0cebe\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-shttb" Aug 04 09:45:25.427497 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.427480 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a6865d23-1fea-4c03-87d9-b6cba1b0cebe-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bb649b865-shttb\" (UID: \"a6865d23-1fea-4c03-87d9-b6cba1b0cebe\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-shttb" Aug 04 09:45:25.427544 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.427505 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a6865d23-1fea-4c03-87d9-b6cba1b0cebe-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bb649b865-shttb\" (UID: \"a6865d23-1fea-4c03-87d9-b6cba1b0cebe\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-shttb" Aug 04 09:45:25.456696 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.456663 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcxwl\" (UniqueName: \"kubernetes.io/projected/a6865d23-1fea-4c03-87d9-b6cba1b0cebe-kube-api-access-wcxwl\") pod \"opendatahub-operator-controller-manager-7bb649b865-shttb\" (UID: \"a6865d23-1fea-4c03-87d9-b6cba1b0cebe\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-shttb" Aug 04 09:45:25.544577 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.544541 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:25.556692 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.556664 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-shttb" Aug 04 09:45:25.713139 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:25.713111 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bb649b865-shttb"] Aug 04 09:45:25.715272 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:45:25.715248 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6865d23_1fea_4c03_87d9_b6cba1b0cebe.slice/crio-94680bf69ae399c1ead134a254b1340339da2b6ae87ac2c65d96850729b481c6 WatchSource:0}: Error finding container 94680bf69ae399c1ead134a254b1340339da2b6ae87ac2c65d96850729b481c6: Status 404 returned error can't find the container with id 94680bf69ae399c1ead134a254b1340339da2b6ae87ac2c65d96850729b481c6 Aug 04 09:45:26.121853 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:26.121755 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-shttb" event={"ID":"a6865d23-1fea-4c03-87d9-b6cba1b0cebe","Type":"ContainerStarted","Data":"94680bf69ae399c1ead134a254b1340339da2b6ae87ac2c65d96850729b481c6"} Aug 04 09:45:26.218501 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:26.218466 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:26.307814 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:26.307770 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:29.139033 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:29.138992 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-shttb" event={"ID":"a6865d23-1fea-4c03-87d9-b6cba1b0cebe","Type":"ContainerStarted","Data":"63357c279a18d95200786c4e1f0b17fce363ab98b9b41b4b8793f031c1f2e7ec"} Aug 04 09:45:30.051652 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:30.051556 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:30.145446 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:30.145409 2572 generic.go:356] "Generic (PLEG): container finished" podID="a6865d23-1fea-4c03-87d9-b6cba1b0cebe" containerID="63357c279a18d95200786c4e1f0b17fce363ab98b9b41b4b8793f031c1f2e7ec" exitCode=0 Aug 04 09:45:30.145837 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:30.145503 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-shttb" event={"ID":"a6865d23-1fea-4c03-87d9-b6cba1b0cebe","Type":"ContainerDied","Data":"63357c279a18d95200786c4e1f0b17fce363ab98b9b41b4b8793f031c1f2e7ec"} Aug 04 09:45:31.151521 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:31.151483 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-shttb" event={"ID":"a6865d23-1fea-4c03-87d9-b6cba1b0cebe","Type":"ContainerStarted","Data":"0b1e67c40a08f5f1b0bdb61d23e1ffc6cd0482c5a7682a4f2e2fa72cdb3d13ce"} Aug 04 09:45:31.151913 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:31.151602 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-shttb" Aug 04 09:45:31.188851 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:31.188766 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-shttb" podStartSLOduration=1.7283736790000002 podStartE2EDuration="6.18874317s" totalImagesPullingTime="3.373905879s" totalInitContainerRuntime="1.086463612s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:45:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:45:31.184671435 +0000 UTC m=+299.928805855" watchObservedRunningTime="2026-08-04 09:45:31.18874317 +0000 UTC m=+299.932877589" Aug 04 09:45:31.632347 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:31.632257 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sdsr_7858cfca-73db-4ee1-9882-c13444f112a5/ovn-acl-logging/0.log" Aug 04 09:45:31.635190 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:31.635167 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sdsr_7858cfca-73db-4ee1-9882-c13444f112a5/ovn-acl-logging/0.log" Aug 04 09:45:31.638828 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:31.638789 2572 kubelet.go:1648] "Image garbage collection succeeded" Aug 04 09:45:42.158782 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:42.158746 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7bb649b865-shttb" Aug 04 09:45:45.516583 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:45.516543 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:47.759245 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:47.759209 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:51.930170 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:51.930123 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:52.054328 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.054293 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-n28ms"] Aug 04 09:45:52.058575 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.058553 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-b474cb6fc-b764w"] Aug 04 09:45:52.058692 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.058670 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-n28ms" Aug 04 09:45:52.061519 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.061495 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-b474cb6fc-b764w" Aug 04 09:45:52.063695 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.063675 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-xwxxf\"" Aug 04 09:45:52.063911 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.063894 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Aug 04 09:45:52.064149 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.064106 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Aug 04 09:45:52.064912 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.064893 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Aug 04 09:45:52.065015 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.064915 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-w9mff\"" Aug 04 09:45:52.077031 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.077000 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-n28ms"] Aug 04 09:45:52.082709 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.082678 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-b474cb6fc-b764w"] Aug 04 09:45:52.098291 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.098248 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:52.149934 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.149887 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ec790de-ee7b-451e-be4c-82fc33248f9b-tmp\") pod \"kube-auth-proxy-b474cb6fc-b764w\" (UID: \"9ec790de-ee7b-451e-be4c-82fc33248f9b\") " pod="openshift-ingress/kube-auth-proxy-b474cb6fc-b764w" Aug 04 09:45:52.150112 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.149952 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcdz5\" (UniqueName: \"kubernetes.io/projected/8dfa4811-aea8-4339-b5f1-d288941e3186-kube-api-access-lcdz5\") pod \"odh-model-controller-7998c67dc4-n28ms\" (UID: \"8dfa4811-aea8-4339-b5f1-d288941e3186\") " pod="opendatahub/odh-model-controller-7998c67dc4-n28ms" Aug 04 09:45:52.150112 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.150021 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkq5f\" (UniqueName: \"kubernetes.io/projected/9ec790de-ee7b-451e-be4c-82fc33248f9b-kube-api-access-zkq5f\") pod \"kube-auth-proxy-b474cb6fc-b764w\" (UID: \"9ec790de-ee7b-451e-be4c-82fc33248f9b\") " pod="openshift-ingress/kube-auth-proxy-b474cb6fc-b764w" Aug 04 09:45:52.150185 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.150114 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8dfa4811-aea8-4339-b5f1-d288941e3186-cert\") pod \"odh-model-controller-7998c67dc4-n28ms\" (UID: \"8dfa4811-aea8-4339-b5f1-d288941e3186\") " pod="opendatahub/odh-model-controller-7998c67dc4-n28ms" Aug 04 09:45:52.150185 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.150146 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ec790de-ee7b-451e-be4c-82fc33248f9b-tls-certs\") pod \"kube-auth-proxy-b474cb6fc-b764w\" (UID: \"9ec790de-ee7b-451e-be4c-82fc33248f9b\") " pod="openshift-ingress/kube-auth-proxy-b474cb6fc-b764w" Aug 04 09:45:52.215817 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.215751 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:52.250528 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.250494 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8dfa4811-aea8-4339-b5f1-d288941e3186-cert\") pod \"odh-model-controller-7998c67dc4-n28ms\" (UID: \"8dfa4811-aea8-4339-b5f1-d288941e3186\") " pod="opendatahub/odh-model-controller-7998c67dc4-n28ms" Aug 04 09:45:52.250528 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.250533 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ec790de-ee7b-451e-be4c-82fc33248f9b-tls-certs\") pod \"kube-auth-proxy-b474cb6fc-b764w\" (UID: \"9ec790de-ee7b-451e-be4c-82fc33248f9b\") " pod="openshift-ingress/kube-auth-proxy-b474cb6fc-b764w" Aug 04 09:45:52.250755 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.250562 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ec790de-ee7b-451e-be4c-82fc33248f9b-tmp\") pod \"kube-auth-proxy-b474cb6fc-b764w\" (UID: \"9ec790de-ee7b-451e-be4c-82fc33248f9b\") " pod="openshift-ingress/kube-auth-proxy-b474cb6fc-b764w" Aug 04 09:45:52.250755 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.250599 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcdz5\" (UniqueName: \"kubernetes.io/projected/8dfa4811-aea8-4339-b5f1-d288941e3186-kube-api-access-lcdz5\") pod \"odh-model-controller-7998c67dc4-n28ms\" (UID: \"8dfa4811-aea8-4339-b5f1-d288941e3186\") " pod="opendatahub/odh-model-controller-7998c67dc4-n28ms" Aug 04 09:45:52.250755 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:45:52.250635 2572 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Aug 04 09:45:52.250755 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:45:52.250706 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dfa4811-aea8-4339-b5f1-d288941e3186-cert podName:8dfa4811-aea8-4339-b5f1-d288941e3186 nodeName:}" failed. No retries permitted until 2026-08-04 09:45:52.750682433 +0000 UTC m=+321.494816850 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8dfa4811-aea8-4339-b5f1-d288941e3186-cert") pod "odh-model-controller-7998c67dc4-n28ms" (UID: "8dfa4811-aea8-4339-b5f1-d288941e3186") : secret "odh-model-controller-webhook-cert" not found Aug 04 09:45:52.250755 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.250729 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkq5f\" (UniqueName: \"kubernetes.io/projected/9ec790de-ee7b-451e-be4c-82fc33248f9b-kube-api-access-zkq5f\") pod \"kube-auth-proxy-b474cb6fc-b764w\" (UID: \"9ec790de-ee7b-451e-be4c-82fc33248f9b\") " pod="openshift-ingress/kube-auth-proxy-b474cb6fc-b764w" Aug 04 09:45:52.252859 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.252831 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ec790de-ee7b-451e-be4c-82fc33248f9b-tmp\") pod \"kube-auth-proxy-b474cb6fc-b764w\" (UID: \"9ec790de-ee7b-451e-be4c-82fc33248f9b\") " pod="openshift-ingress/kube-auth-proxy-b474cb6fc-b764w" Aug 04 09:45:52.252988 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.252972 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ec790de-ee7b-451e-be4c-82fc33248f9b-tls-certs\") pod \"kube-auth-proxy-b474cb6fc-b764w\" (UID: \"9ec790de-ee7b-451e-be4c-82fc33248f9b\") " pod="openshift-ingress/kube-auth-proxy-b474cb6fc-b764w" Aug 04 09:45:52.264084 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.264054 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkq5f\" (UniqueName: \"kubernetes.io/projected/9ec790de-ee7b-451e-be4c-82fc33248f9b-kube-api-access-zkq5f\") pod \"kube-auth-proxy-b474cb6fc-b764w\" (UID: \"9ec790de-ee7b-451e-be4c-82fc33248f9b\") " pod="openshift-ingress/kube-auth-proxy-b474cb6fc-b764w" Aug 04 09:45:52.265439 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.265411 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcdz5\" (UniqueName: \"kubernetes.io/projected/8dfa4811-aea8-4339-b5f1-d288941e3186-kube-api-access-lcdz5\") pod \"odh-model-controller-7998c67dc4-n28ms\" (UID: \"8dfa4811-aea8-4339-b5f1-d288941e3186\") " pod="opendatahub/odh-model-controller-7998c67dc4-n28ms" Aug 04 09:45:52.318999 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.318961 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:52.376332 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.376295 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-b474cb6fc-b764w" Aug 04 09:45:52.439232 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.439190 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:52.524127 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.523978 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-b474cb6fc-b764w"] Aug 04 09:45:52.526433 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:45:52.526406 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ec790de_ee7b_451e_be4c_82fc33248f9b.slice/crio-72ce0d797a0c9766f80ebfbbec25d3feee13f3adc1d1561a655a4451989d59bb WatchSource:0}: Error finding container 72ce0d797a0c9766f80ebfbbec25d3feee13f3adc1d1561a655a4451989d59bb: Status 404 returned error can't find the container with id 72ce0d797a0c9766f80ebfbbec25d3feee13f3adc1d1561a655a4451989d59bb Aug 04 09:45:52.528345 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.528325 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 09:45:52.754970 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.754925 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8dfa4811-aea8-4339-b5f1-d288941e3186-cert\") pod \"odh-model-controller-7998c67dc4-n28ms\" (UID: \"8dfa4811-aea8-4339-b5f1-d288941e3186\") " pod="opendatahub/odh-model-controller-7998c67dc4-n28ms" Aug 04 09:45:52.757330 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.757306 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8dfa4811-aea8-4339-b5f1-d288941e3186-cert\") pod \"odh-model-controller-7998c67dc4-n28ms\" (UID: \"8dfa4811-aea8-4339-b5f1-d288941e3186\") " pod="opendatahub/odh-model-controller-7998c67dc4-n28ms" Aug 04 09:45:52.970060 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:52.970006 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-n28ms" Aug 04 09:45:53.163198 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:53.163123 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-n28ms"] Aug 04 09:45:53.258461 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:53.258369 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-n28ms" event={"ID":"8dfa4811-aea8-4339-b5f1-d288941e3186","Type":"ContainerStarted","Data":"097d2dda07456b2f8d3d5845c278d11aa85705dc94119d8f82a7804112cac17d"} Aug 04 09:45:53.260091 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:53.260056 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-b474cb6fc-b764w" event={"ID":"9ec790de-ee7b-451e-be4c-82fc33248f9b","Type":"ContainerStarted","Data":"72ce0d797a0c9766f80ebfbbec25d3feee13f3adc1d1561a655a4451989d59bb"} Aug 04 09:45:53.309527 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:53.309483 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fxnjm"] Aug 04 09:45:53.313714 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:53.313681 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-fxnjm" Aug 04 09:45:53.318131 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:53.318106 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Aug 04 09:45:53.338328 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:53.338281 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:53.339665 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:53.339637 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fxnjm"] Aug 04 09:45:53.362500 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:53.362462 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bad016c6-1c85-4855-8b2e-a9772ecd302a-tls-certs\") pod \"model-serving-api-54d89bd5fc-fxnjm\" (UID: \"bad016c6-1c85-4855-8b2e-a9772ecd302a\") " pod="opendatahub/model-serving-api-54d89bd5fc-fxnjm" Aug 04 09:45:53.362500 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:53.362519 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b52jv\" (UniqueName: \"kubernetes.io/projected/bad016c6-1c85-4855-8b2e-a9772ecd302a-kube-api-access-b52jv\") pod \"model-serving-api-54d89bd5fc-fxnjm\" (UID: \"bad016c6-1c85-4855-8b2e-a9772ecd302a\") " pod="opendatahub/model-serving-api-54d89bd5fc-fxnjm" Aug 04 09:45:53.463302 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:53.463258 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bad016c6-1c85-4855-8b2e-a9772ecd302a-tls-certs\") pod \"model-serving-api-54d89bd5fc-fxnjm\" (UID: \"bad016c6-1c85-4855-8b2e-a9772ecd302a\") " pod="opendatahub/model-serving-api-54d89bd5fc-fxnjm" Aug 04 09:45:53.463510 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:53.463308 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b52jv\" (UniqueName: \"kubernetes.io/projected/bad016c6-1c85-4855-8b2e-a9772ecd302a-kube-api-access-b52jv\") pod \"model-serving-api-54d89bd5fc-fxnjm\" (UID: \"bad016c6-1c85-4855-8b2e-a9772ecd302a\") " pod="opendatahub/model-serving-api-54d89bd5fc-fxnjm" Aug 04 09:45:53.469671 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:53.469590 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bad016c6-1c85-4855-8b2e-a9772ecd302a-tls-certs\") pod \"model-serving-api-54d89bd5fc-fxnjm\" (UID: \"bad016c6-1c85-4855-8b2e-a9772ecd302a\") " pod="opendatahub/model-serving-api-54d89bd5fc-fxnjm" Aug 04 09:45:53.477312 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:53.477239 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:53.480563 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:53.480500 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b52jv\" (UniqueName: \"kubernetes.io/projected/bad016c6-1c85-4855-8b2e-a9772ecd302a-kube-api-access-b52jv\") pod \"model-serving-api-54d89bd5fc-fxnjm\" (UID: \"bad016c6-1c85-4855-8b2e-a9772ecd302a\") " pod="opendatahub/model-serving-api-54d89bd5fc-fxnjm" Aug 04 09:45:53.629228 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:53.629132 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-fxnjm" Aug 04 09:45:53.815069 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:53.815031 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fxnjm"] Aug 04 09:45:53.819512 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:45:53.819439 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbad016c6_1c85_4855_8b2e_a9772ecd302a.slice/crio-381622643df9e13684918eabe64b2a49ec90917c4ea6d9e59a394bf8843bf2de WatchSource:0}: Error finding container 381622643df9e13684918eabe64b2a49ec90917c4ea6d9e59a394bf8843bf2de: Status 404 returned error can't find the container with id 381622643df9e13684918eabe64b2a49ec90917c4ea6d9e59a394bf8843bf2de Aug 04 09:45:54.269651 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.269564 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-fxnjm" event={"ID":"bad016c6-1c85-4855-8b2e-a9772ecd302a","Type":"ContainerStarted","Data":"381622643df9e13684918eabe64b2a49ec90917c4ea6d9e59a394bf8843bf2de"} Aug 04 09:45:54.420273 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.420238 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq"] Aug 04 09:45:54.422019 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.421895 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:54.425149 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.425105 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" Aug 04 09:45:54.429617 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.429591 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Aug 04 09:45:54.429880 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.429856 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-rqntt\"" Aug 04 09:45:54.430088 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.430070 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Aug 04 09:45:54.430307 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.430289 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Aug 04 09:45:54.430571 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.430551 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Aug 04 09:45:54.454660 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.454616 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq"] Aug 04 09:45:54.474148 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.474108 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-bdqhq\" (UID: \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" Aug 04 09:45:54.474330 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.474181 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-bdqhq\" (UID: \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" Aug 04 09:45:54.474330 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.474215 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxmbl\" (UniqueName: \"kubernetes.io/projected/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-kube-api-access-cxmbl\") pod \"istiod-openshift-gateway-7b7995d5cd-bdqhq\" (UID: \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" Aug 04 09:45:54.474330 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.474257 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-bdqhq\" (UID: \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" Aug 04 09:45:54.474330 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.474288 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-bdqhq\" (UID: \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" Aug 04 09:45:54.474330 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.474314 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-bdqhq\" (UID: \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" Aug 04 09:45:54.474587 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.474364 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-bdqhq\" (UID: \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" Aug 04 09:45:54.576665 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.575225 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-bdqhq\" (UID: \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" Aug 04 09:45:54.576665 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.575323 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-bdqhq\" (UID: \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" Aug 04 09:45:54.576665 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.575368 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-bdqhq\" (UID: \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" Aug 04 09:45:54.576665 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.575399 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxmbl\" (UniqueName: \"kubernetes.io/projected/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-kube-api-access-cxmbl\") pod \"istiod-openshift-gateway-7b7995d5cd-bdqhq\" (UID: \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" Aug 04 09:45:54.576665 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.575434 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-bdqhq\" (UID: \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" Aug 04 09:45:54.576665 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.575465 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-bdqhq\" (UID: \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" Aug 04 09:45:54.576665 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.575493 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-bdqhq\" (UID: \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" Aug 04 09:45:54.576665 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.576556 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-bdqhq\" (UID: \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" Aug 04 09:45:54.578683 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.578630 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-bdqhq\" (UID: \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" Aug 04 09:45:54.581763 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.581712 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-bdqhq\" (UID: \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" Aug 04 09:45:54.582248 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.582221 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-bdqhq\" (UID: \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" Aug 04 09:45:54.584052 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.584028 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-bdqhq\" (UID: \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" Aug 04 09:45:54.592893 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.592826 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-bdqhq\" (UID: \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" Aug 04 09:45:54.594688 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.594657 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxmbl\" (UniqueName: \"kubernetes.io/projected/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-kube-api-access-cxmbl\") pod \"istiod-openshift-gateway-7b7995d5cd-bdqhq\" (UID: \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" Aug 04 09:45:54.740814 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.740760 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" Aug 04 09:45:54.744001 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:54.743963 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:56.350224 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:56.350187 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq"] Aug 04 09:45:56.912966 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:45:56.912917 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b112f70_d402_4483_8a67_fcc0cb5b8dcd.slice/crio-1b00bdece7b9aaa8112cbf86fc5707d9de3c123839735b4f2b7be189cb9ed428 WatchSource:0}: Error finding container 1b00bdece7b9aaa8112cbf86fc5707d9de3c123839735b4f2b7be189cb9ed428: Status 404 returned error can't find the container with id 1b00bdece7b9aaa8112cbf86fc5707d9de3c123839735b4f2b7be189cb9ed428 Aug 04 09:45:57.290539 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:57.290491 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" event={"ID":"1b112f70-d402-4483-8a67-fcc0cb5b8dcd","Type":"ContainerStarted","Data":"1b00bdece7b9aaa8112cbf86fc5707d9de3c123839735b4f2b7be189cb9ed428"} Aug 04 09:45:57.494284 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:57.494248 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:45:58.298356 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:58.298313 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-n28ms" event={"ID":"8dfa4811-aea8-4339-b5f1-d288941e3186","Type":"ContainerStarted","Data":"6d75e09182d5c3350e12b66d17f5bd957f9415fae9b00ab065a7f02ee570b9e1"} Aug 04 09:45:58.298551 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:58.298412 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-n28ms" Aug 04 09:45:58.299971 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:58.299941 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-fxnjm" event={"ID":"bad016c6-1c85-4855-8b2e-a9772ecd302a","Type":"ContainerStarted","Data":"2be215f3c0c2bc95d7840c4560f65fdd557d7a2d31906c46c3848172c3299556"} Aug 04 09:45:58.300126 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:58.300025 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-fxnjm" Aug 04 09:45:58.301518 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:58.301495 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-b474cb6fc-b764w" event={"ID":"9ec790de-ee7b-451e-be4c-82fc33248f9b","Type":"ContainerStarted","Data":"0030c27fb8aafc08646ba72e277ec16bdc649ca1dc882e5074c5df52b6561337"} Aug 04 09:45:58.326821 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:58.326746 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-n28ms" podStartSLOduration=1.981076475 podStartE2EDuration="6.326729965s" totalImagesPullingTime="4.34565349s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:45:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:45:58.325234085 +0000 UTC m=+327.069368503" watchObservedRunningTime="2026-08-04 09:45:58.326729965 +0000 UTC m=+327.070864384" Aug 04 09:45:58.369103 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:58.369039 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-fxnjm" podStartSLOduration=1.7240550049999999 podStartE2EDuration="5.369022449s" totalImagesPullingTime="3.644967444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:45:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:45:58.36451702 +0000 UTC m=+327.108651452" watchObservedRunningTime="2026-08-04 09:45:58.369022449 +0000 UTC m=+327.113156868" Aug 04 09:45:58.399064 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:45:58.399002 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-b474cb6fc-b764w" podStartSLOduration=1.460493358 podStartE2EDuration="6.398984073s" totalImagesPullingTime="4.938490715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:45:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:45:58.396318212 +0000 UTC m=+327.140452633" watchObservedRunningTime="2026-08-04 09:45:58.398984073 +0000 UTC m=+327.143118491" Aug 04 09:46:00.211870 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:00.211793 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 09:46:00.212165 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:00.211906 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 09:46:00.313666 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:00.313625 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" event={"ID":"1b112f70-d402-4483-8a67-fcc0cb5b8dcd","Type":"ContainerStarted","Data":"592e20e3a2f0591c80da66edffc7334c410c21280db2d61f753077af309b2cff"} Aug 04 09:46:00.313905 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:00.313718 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" Aug 04 09:46:01.320682 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:01.320647 2572 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-bdqhq container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 09:46:01.321087 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:01.320706 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" podUID="1b112f70-d402-4483-8a67-fcc0cb5b8dcd" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 09:46:01.746424 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:01.746389 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:46:02.134863 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:02.134749 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:46:02.724178 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:02.724144 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:46:02.962040 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:02.962004 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:46:04.005482 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.005411 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" podStartSLOduration=6.712739343 podStartE2EDuration="10.005386827s" totalImagesPullingTime="3.292647484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:45:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:46:00.389165572 +0000 UTC m=+329.133299991" watchObservedRunningTime="2026-08-04 09:46:04.005386827 +0000 UTC m=+332.749521247" Aug 04 09:46:04.007629 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.007603 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc"] Aug 04 09:46:04.014881 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.014860 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:04.019197 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.019178 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-vlc24\"" Aug 04 09:46:04.170301 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.170258 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc\" (UID: \"2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:04.170474 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.170322 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc\" (UID: \"2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:04.170474 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.170352 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc\" (UID: \"2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:04.170474 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.170384 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc\" (UID: \"2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:04.170474 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.170428 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc\" (UID: \"2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:04.170474 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.170456 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc\" (UID: \"2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:04.170720 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.170476 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmttw\" (UniqueName: \"kubernetes.io/projected/2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba-kube-api-access-hmttw\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc\" (UID: \"2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:04.170720 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.170566 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc\" (UID: \"2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:04.170720 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.170617 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc\" (UID: \"2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:04.256348 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.256267 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:46:04.271207 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.271173 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc\" (UID: \"2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:04.271342 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.271224 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc\" (UID: \"2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:04.271342 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.271256 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmttw\" (UniqueName: \"kubernetes.io/projected/2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba-kube-api-access-hmttw\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc\" (UID: \"2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:04.271342 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.271294 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc\" (UID: \"2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:04.271342 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.271327 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc\" (UID: \"2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:04.271558 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.271526 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc\" (UID: \"2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:04.271926 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.271628 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc\" (UID: \"2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:04.271926 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.271660 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc\" (UID: \"2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:04.271926 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.271672 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc\" (UID: \"2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:04.271926 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.271676 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc\" (UID: \"2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:04.271926 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.271712 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc\" (UID: \"2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:04.271926 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.271745 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc\" (UID: \"2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:04.272240 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.271967 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc\" (UID: \"2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:04.272320 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.272301 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc\" (UID: \"2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:04.273660 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.273637 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc\" (UID: \"2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:04.273904 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.273887 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc\" (UID: \"2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:04.319965 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.319930 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" Aug 04 09:46:04.374908 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.374867 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc"] Aug 04 09:46:04.386557 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.386515 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmttw\" (UniqueName: \"kubernetes.io/projected/2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba-kube-api-access-hmttw\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc\" (UID: \"2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:04.398610 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.398580 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc\" (UID: \"2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:04.625188 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.625097 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:04.677972 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.677563 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:46:04.877125 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:04.877024 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc"] Aug 04 09:46:04.879822 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:46:04.879784 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e0c7407_0a0a_4791_8b8c_f3d2609ad8ba.slice/crio-42cccffdcbf9bb0b9093a89ed0decd544fe76039a005f76427091d53efcd4331 WatchSource:0}: Error finding container 42cccffdcbf9bb0b9093a89ed0decd544fe76039a005f76427091d53efcd4331: Status 404 returned error can't find the container with id 42cccffdcbf9bb0b9093a89ed0decd544fe76039a005f76427091d53efcd4331 Aug 04 09:46:05.017090 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:05.017001 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:46:05.339449 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:05.339410 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" event={"ID":"2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba","Type":"ContainerStarted","Data":"42cccffdcbf9bb0b9093a89ed0decd544fe76039a005f76427091d53efcd4331"} Aug 04 09:46:06.764471 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:06.764430 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:46:07.015826 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:07.015721 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:46:07.856323 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:07.856286 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 09:46:07.856718 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:07.856371 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 09:46:07.856718 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:07.856419 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 09:46:08.365019 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:08.364974 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" event={"ID":"2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba","Type":"ContainerStarted","Data":"620d91382799a9397721e9baa19c6896e161b342bf8901b4640559bc927a2427"} Aug 04 09:46:08.464933 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:08.464878 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" podStartSLOduration=2.49076145 podStartE2EDuration="5.464860674s" totalImagesPullingTime="2.974099224s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:46:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:46:08.45382793 +0000 UTC m=+337.197962347" watchObservedRunningTime="2026-08-04 09:46:08.464860674 +0000 UTC m=+337.208995092" Aug 04 09:46:08.625750 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:08.625649 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:08.630617 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:08.630589 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:09.310364 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:09.310339 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-n28ms" Aug 04 09:46:09.312247 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:09.312228 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-fxnjm" Aug 04 09:46:09.370015 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:09.369983 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:09.370866 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:09.370850 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc" Aug 04 09:46:09.603967 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:09.603870 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:46:09.727004 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:09.726971 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-cx42r"] Aug 04 09:46:09.730326 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:09.730304 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-cx42r" Aug 04 09:46:09.747711 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:09.747685 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-c4hxm\"" Aug 04 09:46:09.747890 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:09.747860 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Aug 04 09:46:09.779331 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:09.779300 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-cx42r"] Aug 04 09:46:09.823915 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:09.823880 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4dnq\" (UniqueName: \"kubernetes.io/projected/526d3ebd-1f9f-49bc-8fcb-08a3ee209794-kube-api-access-m4dnq\") pod \"kserve-controller-manager-d6f4b65b8-cx42r\" (UID: \"526d3ebd-1f9f-49bc-8fcb-08a3ee209794\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-cx42r" Aug 04 09:46:09.824087 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:09.823946 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/526d3ebd-1f9f-49bc-8fcb-08a3ee209794-cert\") pod \"kserve-controller-manager-d6f4b65b8-cx42r\" (UID: \"526d3ebd-1f9f-49bc-8fcb-08a3ee209794\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-cx42r" Aug 04 09:46:09.916113 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:09.916023 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:46:09.925240 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:09.925207 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/526d3ebd-1f9f-49bc-8fcb-08a3ee209794-cert\") pod \"kserve-controller-manager-d6f4b65b8-cx42r\" (UID: \"526d3ebd-1f9f-49bc-8fcb-08a3ee209794\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-cx42r" Aug 04 09:46:09.925373 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:09.925296 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4dnq\" (UniqueName: \"kubernetes.io/projected/526d3ebd-1f9f-49bc-8fcb-08a3ee209794-kube-api-access-m4dnq\") pod \"kserve-controller-manager-d6f4b65b8-cx42r\" (UID: \"526d3ebd-1f9f-49bc-8fcb-08a3ee209794\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-cx42r" Aug 04 09:46:09.927713 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:09.927690 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/526d3ebd-1f9f-49bc-8fcb-08a3ee209794-cert\") pod \"kserve-controller-manager-d6f4b65b8-cx42r\" (UID: \"526d3ebd-1f9f-49bc-8fcb-08a3ee209794\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-cx42r" Aug 04 09:46:09.991997 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:09.991968 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4dnq\" (UniqueName: \"kubernetes.io/projected/526d3ebd-1f9f-49bc-8fcb-08a3ee209794-kube-api-access-m4dnq\") pod \"kserve-controller-manager-d6f4b65b8-cx42r\" (UID: \"526d3ebd-1f9f-49bc-8fcb-08a3ee209794\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-cx42r" Aug 04 09:46:10.022784 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:10.022741 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6jrrz"] Aug 04 09:46:10.026968 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:10.026376 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6jrrz" Aug 04 09:46:10.040684 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:10.040658 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-cx42r" Aug 04 09:46:10.045971 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:10.045949 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-9hrkd\"" Aug 04 09:46:10.057077 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:10.056953 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Aug 04 09:46:10.058449 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:10.058419 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6jrrz"] Aug 04 09:46:10.129613 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:10.129545 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c2413b7-aca2-40ae-b13a-b4d193968c4a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6jrrz\" (UID: \"7c2413b7-aca2-40ae-b13a-b4d193968c4a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6jrrz" Aug 04 09:46:10.129771 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:10.129676 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7448n\" (UniqueName: \"kubernetes.io/projected/7c2413b7-aca2-40ae-b13a-b4d193968c4a-kube-api-access-7448n\") pod \"llmisvc-controller-manager-777c6bd7d5-6jrrz\" (UID: \"7c2413b7-aca2-40ae-b13a-b4d193968c4a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6jrrz" Aug 04 09:46:10.231225 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:10.231188 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7448n\" (UniqueName: \"kubernetes.io/projected/7c2413b7-aca2-40ae-b13a-b4d193968c4a-kube-api-access-7448n\") pod \"llmisvc-controller-manager-777c6bd7d5-6jrrz\" (UID: \"7c2413b7-aca2-40ae-b13a-b4d193968c4a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6jrrz" Aug 04 09:46:10.231396 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:10.231302 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c2413b7-aca2-40ae-b13a-b4d193968c4a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6jrrz\" (UID: \"7c2413b7-aca2-40ae-b13a-b4d193968c4a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6jrrz" Aug 04 09:46:10.233617 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:10.233596 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c2413b7-aca2-40ae-b13a-b4d193968c4a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6jrrz\" (UID: \"7c2413b7-aca2-40ae-b13a-b4d193968c4a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6jrrz" Aug 04 09:46:10.385851 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:10.385821 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7448n\" (UniqueName: \"kubernetes.io/projected/7c2413b7-aca2-40ae-b13a-b4d193968c4a-kube-api-access-7448n\") pod \"llmisvc-controller-manager-777c6bd7d5-6jrrz\" (UID: \"7c2413b7-aca2-40ae-b13a-b4d193968c4a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6jrrz" Aug 04 09:46:10.414956 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:10.414922 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-cx42r"] Aug 04 09:46:10.418033 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:46:10.418002 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod526d3ebd_1f9f_49bc_8fcb_08a3ee209794.slice/crio-dd11525020bd9121fd5266e990ce56181fb6a7efd45fb5fa3278e4b416025aae WatchSource:0}: Error finding container dd11525020bd9121fd5266e990ce56181fb6a7efd45fb5fa3278e4b416025aae: Status 404 returned error can't find the container with id dd11525020bd9121fd5266e990ce56181fb6a7efd45fb5fa3278e4b416025aae Aug 04 09:46:10.643075 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:10.642979 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6jrrz" Aug 04 09:46:10.907159 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:10.907070 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:46:10.968123 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:10.968092 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6jrrz"] Aug 04 09:46:11.017082 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:46:11.017044 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7c2413b7_aca2_40ae_b13a_b4d193968c4a.slice/crio-9fe702da09026f015e296bb0bc3cbbd419e2af1b6922ee9bf2e2111970fdcc85 WatchSource:0}: Error finding container 9fe702da09026f015e296bb0bc3cbbd419e2af1b6922ee9bf2e2111970fdcc85: Status 404 returned error can't find the container with id 9fe702da09026f015e296bb0bc3cbbd419e2af1b6922ee9bf2e2111970fdcc85 Aug 04 09:46:11.200916 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:11.200879 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq"] Aug 04 09:46:11.201147 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:11.201121 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" podUID="1b112f70-d402-4483-8a67-fcc0cb5b8dcd" containerName="discovery" containerID="cri-o://592e20e3a2f0591c80da66edffc7334c410c21280db2d61f753077af309b2cff" gracePeriod=30 Aug 04 09:46:11.381375 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:11.381308 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-cx42r" event={"ID":"526d3ebd-1f9f-49bc-8fcb-08a3ee209794","Type":"ContainerStarted","Data":"dd11525020bd9121fd5266e990ce56181fb6a7efd45fb5fa3278e4b416025aae"} Aug 04 09:46:11.382990 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:11.382957 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6jrrz" event={"ID":"7c2413b7-aca2-40ae-b13a-b4d193968c4a","Type":"ContainerStarted","Data":"9fe702da09026f015e296bb0bc3cbbd419e2af1b6922ee9bf2e2111970fdcc85"} Aug 04 09:46:11.529949 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:11.529905 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:46:11.585587 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:11.585549 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" Aug 04 09:46:11.745613 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:11.745517 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxmbl\" (UniqueName: \"kubernetes.io/projected/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-kube-api-access-cxmbl\") pod \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\" (UID: \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\") " Aug 04 09:46:11.745780 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:11.745613 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-istio-kubeconfig\") pod \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\" (UID: \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\") " Aug 04 09:46:11.745780 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:11.745643 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-istio-csr-dns-cert\") pod \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\" (UID: \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\") " Aug 04 09:46:11.745780 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:11.745673 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-istio-csr-ca-configmap\") pod \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\" (UID: \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\") " Aug 04 09:46:11.745780 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:11.745703 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-istio-token\") pod \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\" (UID: \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\") " Aug 04 09:46:11.745780 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:11.745731 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-cacerts\") pod \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\" (UID: \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\") " Aug 04 09:46:11.745780 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:11.745764 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-local-certs\") pod \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\" (UID: \"1b112f70-d402-4483-8a67-fcc0cb5b8dcd\") " Aug 04 09:46:11.746237 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:11.746209 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "1b112f70-d402-4483-8a67-fcc0cb5b8dcd" (UID: "1b112f70-d402-4483-8a67-fcc0cb5b8dcd"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:46:11.749940 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:11.749890 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-cacerts" (OuterVolumeSpecName: "cacerts") pod "1b112f70-d402-4483-8a67-fcc0cb5b8dcd" (UID: "1b112f70-d402-4483-8a67-fcc0cb5b8dcd"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:46:11.749940 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:11.749908 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-kube-api-access-cxmbl" (OuterVolumeSpecName: "kube-api-access-cxmbl") pod "1b112f70-d402-4483-8a67-fcc0cb5b8dcd" (UID: "1b112f70-d402-4483-8a67-fcc0cb5b8dcd"). InnerVolumeSpecName "kube-api-access-cxmbl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:46:11.749940 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:11.749939 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-istio-token" (OuterVolumeSpecName: "istio-token") pod "1b112f70-d402-4483-8a67-fcc0cb5b8dcd" (UID: "1b112f70-d402-4483-8a67-fcc0cb5b8dcd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:46:11.750232 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:11.750131 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "1b112f70-d402-4483-8a67-fcc0cb5b8dcd" (UID: "1b112f70-d402-4483-8a67-fcc0cb5b8dcd"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:46:11.750492 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:11.750463 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "1b112f70-d402-4483-8a67-fcc0cb5b8dcd" (UID: "1b112f70-d402-4483-8a67-fcc0cb5b8dcd"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:46:11.751868 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:11.751810 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-local-certs" (OuterVolumeSpecName: "local-certs") pod "1b112f70-d402-4483-8a67-fcc0cb5b8dcd" (UID: "1b112f70-d402-4483-8a67-fcc0cb5b8dcd"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:46:11.841072 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:11.841034 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:46:11.847308 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:11.847275 2572 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-istio-kubeconfig\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:46:11.847308 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:11.847308 2572 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-istio-csr-dns-cert\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:46:11.847499 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:11.847324 2572 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-istio-csr-ca-configmap\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:46:11.847499 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:11.847336 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-istio-token\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:46:11.847499 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:11.847347 2572 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-cacerts\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:46:11.847499 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:11.847358 2572 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-local-certs\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:46:11.847499 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:11.847368 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxmbl\" (UniqueName: \"kubernetes.io/projected/1b112f70-d402-4483-8a67-fcc0cb5b8dcd-kube-api-access-cxmbl\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:46:12.390608 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:12.390564 2572 generic.go:356] "Generic (PLEG): container finished" podID="1b112f70-d402-4483-8a67-fcc0cb5b8dcd" containerID="592e20e3a2f0591c80da66edffc7334c410c21280db2d61f753077af309b2cff" exitCode=0 Aug 04 09:46:12.390856 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:12.390641 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" Aug 04 09:46:12.390856 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:12.390665 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" event={"ID":"1b112f70-d402-4483-8a67-fcc0cb5b8dcd","Type":"ContainerDied","Data":"592e20e3a2f0591c80da66edffc7334c410c21280db2d61f753077af309b2cff"} Aug 04 09:46:12.390856 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:12.390733 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq" event={"ID":"1b112f70-d402-4483-8a67-fcc0cb5b8dcd","Type":"ContainerDied","Data":"1b00bdece7b9aaa8112cbf86fc5707d9de3c123839735b4f2b7be189cb9ed428"} Aug 04 09:46:12.390856 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:12.390759 2572 scope.go:117] "RemoveContainer" containerID="592e20e3a2f0591c80da66edffc7334c410c21280db2d61f753077af309b2cff" Aug 04 09:46:12.436862 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:12.436825 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq"] Aug 04 09:46:12.450984 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:12.450954 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bdqhq"] Aug 04 09:46:12.911637 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:12.911605 2572 scope.go:117] "RemoveContainer" containerID="592e20e3a2f0591c80da66edffc7334c410c21280db2d61f753077af309b2cff" Aug 04 09:46:12.912007 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:46:12.911984 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"592e20e3a2f0591c80da66edffc7334c410c21280db2d61f753077af309b2cff\": container with ID starting with 592e20e3a2f0591c80da66edffc7334c410c21280db2d61f753077af309b2cff not found: ID does not exist" containerID="592e20e3a2f0591c80da66edffc7334c410c21280db2d61f753077af309b2cff" Aug 04 09:46:12.912058 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:12.912017 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"592e20e3a2f0591c80da66edffc7334c410c21280db2d61f753077af309b2cff"} err="failed to get container status \"592e20e3a2f0591c80da66edffc7334c410c21280db2d61f753077af309b2cff\": rpc error: code = NotFound desc = could not find container \"592e20e3a2f0591c80da66edffc7334c410c21280db2d61f753077af309b2cff\": container with ID starting with 592e20e3a2f0591c80da66edffc7334c410c21280db2d61f753077af309b2cff not found: ID does not exist" Aug 04 09:46:13.778086 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:13.778049 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b112f70-d402-4483-8a67-fcc0cb5b8dcd" path="/var/lib/kubelet/pods/1b112f70-d402-4483-8a67-fcc0cb5b8dcd/volumes" Aug 04 09:46:14.404022 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:14.403984 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-cx42r" event={"ID":"526d3ebd-1f9f-49bc-8fcb-08a3ee209794","Type":"ContainerStarted","Data":"5bde74866f120554fa1c270aaca4f7f94b477b352fbb8e1b0b0c292b35065b3b"} Aug 04 09:46:14.404443 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:14.404083 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-cx42r" Aug 04 09:46:14.405650 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:14.405619 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6jrrz" event={"ID":"7c2413b7-aca2-40ae-b13a-b4d193968c4a","Type":"ContainerStarted","Data":"5d0e14dfa1a6eec07d748519e2b9d74f65d2100641386cffb4dc4ed7e49e95a0"} Aug 04 09:46:14.405782 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:14.405687 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6jrrz" Aug 04 09:46:14.460001 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:14.459950 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-cx42r" podStartSLOduration=2.752950128 podStartE2EDuration="5.459933595s" totalImagesPullingTime="2.706983467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:46:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:46:14.45367734 +0000 UTC m=+343.197811759" watchObservedRunningTime="2026-08-04 09:46:14.459933595 +0000 UTC m=+343.204068014" Aug 04 09:46:14.498951 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:14.498891 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6jrrz" podStartSLOduration=2.678754526 podStartE2EDuration="5.498875624s" totalImagesPullingTime="2.820121098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:46:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:46:14.495523839 +0000 UTC m=+343.239658261" watchObservedRunningTime="2026-08-04 09:46:14.498875624 +0000 UTC m=+343.243010043" Aug 04 09:46:45.413194 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:45.413161 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6jrrz" Aug 04 09:46:45.416223 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:46:45.416202 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-cx42r" Aug 04 09:47:07.555295 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:07.555260 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:07.738932 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:07.738895 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:12.029787 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.029752 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v"] Aug 04 09:47:12.030178 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.030127 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b112f70-d402-4483-8a67-fcc0cb5b8dcd" containerName="discovery" Aug 04 09:47:12.030178 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.030140 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b112f70-d402-4483-8a67-fcc0cb5b8dcd" containerName="discovery" Aug 04 09:47:12.030249 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.030206 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b112f70-d402-4483-8a67-fcc0cb5b8dcd" containerName="discovery" Aug 04 09:47:12.033364 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.033347 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v" Aug 04 09:47:12.036346 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.036318 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Aug 04 09:47:12.036560 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.036383 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Aug 04 09:47:12.036560 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.036416 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-grkw6\"" Aug 04 09:47:12.044581 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.044550 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v"] Aug 04 09:47:12.103627 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.103585 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a69cb15f-3ab7-4b9d-817b-3e177e319c97-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v\" (UID: \"a69cb15f-3ab7-4b9d-817b-3e177e319c97\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v" Aug 04 09:47:12.103627 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.103634 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a69cb15f-3ab7-4b9d-817b-3e177e319c97-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v\" (UID: \"a69cb15f-3ab7-4b9d-817b-3e177e319c97\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v" Aug 04 09:47:12.103923 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.103710 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2ff4\" (UniqueName: \"kubernetes.io/projected/a69cb15f-3ab7-4b9d-817b-3e177e319c97-kube-api-access-d2ff4\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v\" (UID: \"a69cb15f-3ab7-4b9d-817b-3e177e319c97\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v" Aug 04 09:47:12.160383 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.160344 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x"] Aug 04 09:47:12.163877 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.163856 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x" Aug 04 09:47:12.175371 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.175340 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x"] Aug 04 09:47:12.204528 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.204492 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a69cb15f-3ab7-4b9d-817b-3e177e319c97-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v\" (UID: \"a69cb15f-3ab7-4b9d-817b-3e177e319c97\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v" Aug 04 09:47:12.204730 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.204537 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2c2d1ca-7214-44c9-9257-bde0bc98a16a-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x\" (UID: \"c2c2d1ca-7214-44c9-9257-bde0bc98a16a\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x" Aug 04 09:47:12.204730 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.204561 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjxj7\" (UniqueName: \"kubernetes.io/projected/c2c2d1ca-7214-44c9-9257-bde0bc98a16a-kube-api-access-pjxj7\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x\" (UID: \"c2c2d1ca-7214-44c9-9257-bde0bc98a16a\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x" Aug 04 09:47:12.204730 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.204582 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a69cb15f-3ab7-4b9d-817b-3e177e319c97-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v\" (UID: \"a69cb15f-3ab7-4b9d-817b-3e177e319c97\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v" Aug 04 09:47:12.204730 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.204598 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2c2d1ca-7214-44c9-9257-bde0bc98a16a-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x\" (UID: \"c2c2d1ca-7214-44c9-9257-bde0bc98a16a\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x" Aug 04 09:47:12.204730 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.204712 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2ff4\" (UniqueName: \"kubernetes.io/projected/a69cb15f-3ab7-4b9d-817b-3e177e319c97-kube-api-access-d2ff4\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v\" (UID: \"a69cb15f-3ab7-4b9d-817b-3e177e319c97\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v" Aug 04 09:47:12.205043 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.204964 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a69cb15f-3ab7-4b9d-817b-3e177e319c97-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v\" (UID: \"a69cb15f-3ab7-4b9d-817b-3e177e319c97\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v" Aug 04 09:47:12.205043 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.204986 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a69cb15f-3ab7-4b9d-817b-3e177e319c97-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v\" (UID: \"a69cb15f-3ab7-4b9d-817b-3e177e319c97\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v" Aug 04 09:47:12.216557 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.216522 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2ff4\" (UniqueName: \"kubernetes.io/projected/a69cb15f-3ab7-4b9d-817b-3e177e319c97-kube-api-access-d2ff4\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v\" (UID: \"a69cb15f-3ab7-4b9d-817b-3e177e319c97\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v" Aug 04 09:47:12.258556 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.258516 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h"] Aug 04 09:47:12.262177 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.262152 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h" Aug 04 09:47:12.277101 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.277072 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h"] Aug 04 09:47:12.306004 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.305913 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2c2d1ca-7214-44c9-9257-bde0bc98a16a-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x\" (UID: \"c2c2d1ca-7214-44c9-9257-bde0bc98a16a\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x" Aug 04 09:47:12.306004 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.305952 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjxj7\" (UniqueName: \"kubernetes.io/projected/c2c2d1ca-7214-44c9-9257-bde0bc98a16a-kube-api-access-pjxj7\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x\" (UID: \"c2c2d1ca-7214-44c9-9257-bde0bc98a16a\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x" Aug 04 09:47:12.306213 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.306049 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2c2d1ca-7214-44c9-9257-bde0bc98a16a-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x\" (UID: \"c2c2d1ca-7214-44c9-9257-bde0bc98a16a\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x" Aug 04 09:47:12.306213 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.306082 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb26ed9f-a18a-412d-9b72-c3514a7b864a-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h\" (UID: \"fb26ed9f-a18a-412d-9b72-c3514a7b864a\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h" Aug 04 09:47:12.306309 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.306288 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2c2d1ca-7214-44c9-9257-bde0bc98a16a-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x\" (UID: \"c2c2d1ca-7214-44c9-9257-bde0bc98a16a\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x" Aug 04 09:47:12.306351 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.306296 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4wx4\" (UniqueName: \"kubernetes.io/projected/fb26ed9f-a18a-412d-9b72-c3514a7b864a-kube-api-access-j4wx4\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h\" (UID: \"fb26ed9f-a18a-412d-9b72-c3514a7b864a\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h" Aug 04 09:47:12.306388 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.306358 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2c2d1ca-7214-44c9-9257-bde0bc98a16a-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x\" (UID: \"c2c2d1ca-7214-44c9-9257-bde0bc98a16a\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x" Aug 04 09:47:12.306433 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.306416 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb26ed9f-a18a-412d-9b72-c3514a7b864a-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h\" (UID: \"fb26ed9f-a18a-412d-9b72-c3514a7b864a\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h" Aug 04 09:47:12.319922 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.319895 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjxj7\" (UniqueName: \"kubernetes.io/projected/c2c2d1ca-7214-44c9-9257-bde0bc98a16a-kube-api-access-pjxj7\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x\" (UID: \"c2c2d1ca-7214-44c9-9257-bde0bc98a16a\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x" Aug 04 09:47:12.340985 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.340955 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85"] Aug 04 09:47:12.343713 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.343683 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v" Aug 04 09:47:12.344658 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.344640 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85" Aug 04 09:47:12.358052 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.358018 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85"] Aug 04 09:47:12.407158 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.407121 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb26ed9f-a18a-412d-9b72-c3514a7b864a-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h\" (UID: \"fb26ed9f-a18a-412d-9b72-c3514a7b864a\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h" Aug 04 09:47:12.407158 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.407159 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgwxh\" (UniqueName: \"kubernetes.io/projected/6a71d0dc-95ff-4012-9848-cad7679615c9-kube-api-access-cgwxh\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85\" (UID: \"6a71d0dc-95ff-4012-9848-cad7679615c9\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85" Aug 04 09:47:12.407387 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.407196 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a71d0dc-95ff-4012-9848-cad7679615c9-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85\" (UID: \"6a71d0dc-95ff-4012-9848-cad7679615c9\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85" Aug 04 09:47:12.407387 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.407223 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb26ed9f-a18a-412d-9b72-c3514a7b864a-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h\" (UID: \"fb26ed9f-a18a-412d-9b72-c3514a7b864a\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h" Aug 04 09:47:12.407387 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.407243 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a71d0dc-95ff-4012-9848-cad7679615c9-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85\" (UID: \"6a71d0dc-95ff-4012-9848-cad7679615c9\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85" Aug 04 09:47:12.407387 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.407269 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4wx4\" (UniqueName: \"kubernetes.io/projected/fb26ed9f-a18a-412d-9b72-c3514a7b864a-kube-api-access-j4wx4\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h\" (UID: \"fb26ed9f-a18a-412d-9b72-c3514a7b864a\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h" Aug 04 09:47:12.407637 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.407610 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb26ed9f-a18a-412d-9b72-c3514a7b864a-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h\" (UID: \"fb26ed9f-a18a-412d-9b72-c3514a7b864a\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h" Aug 04 09:47:12.407826 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.407788 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb26ed9f-a18a-412d-9b72-c3514a7b864a-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h\" (UID: \"fb26ed9f-a18a-412d-9b72-c3514a7b864a\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h" Aug 04 09:47:12.417858 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.417827 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4wx4\" (UniqueName: \"kubernetes.io/projected/fb26ed9f-a18a-412d-9b72-c3514a7b864a-kube-api-access-j4wx4\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h\" (UID: \"fb26ed9f-a18a-412d-9b72-c3514a7b864a\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h" Aug 04 09:47:12.473957 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.473923 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x" Aug 04 09:47:12.486086 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.486054 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v"] Aug 04 09:47:12.487754 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:47:12.487726 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda69cb15f_3ab7_4b9d_817b_3e177e319c97.slice/crio-77e242021e37d46d156008cead0cf22cef24bf71d4acb9f7a201ed0429a9539c WatchSource:0}: Error finding container 77e242021e37d46d156008cead0cf22cef24bf71d4acb9f7a201ed0429a9539c: Status 404 returned error can't find the container with id 77e242021e37d46d156008cead0cf22cef24bf71d4acb9f7a201ed0429a9539c Aug 04 09:47:12.508584 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.508522 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a71d0dc-95ff-4012-9848-cad7679615c9-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85\" (UID: \"6a71d0dc-95ff-4012-9848-cad7679615c9\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85" Aug 04 09:47:12.508746 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.508616 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a71d0dc-95ff-4012-9848-cad7679615c9-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85\" (UID: \"6a71d0dc-95ff-4012-9848-cad7679615c9\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85" Aug 04 09:47:12.508746 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.508725 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgwxh\" (UniqueName: \"kubernetes.io/projected/6a71d0dc-95ff-4012-9848-cad7679615c9-kube-api-access-cgwxh\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85\" (UID: \"6a71d0dc-95ff-4012-9848-cad7679615c9\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85" Aug 04 09:47:12.508961 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.508937 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a71d0dc-95ff-4012-9848-cad7679615c9-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85\" (UID: \"6a71d0dc-95ff-4012-9848-cad7679615c9\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85" Aug 04 09:47:12.509108 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.509085 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a71d0dc-95ff-4012-9848-cad7679615c9-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85\" (UID: \"6a71d0dc-95ff-4012-9848-cad7679615c9\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85" Aug 04 09:47:12.519414 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.519379 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgwxh\" (UniqueName: \"kubernetes.io/projected/6a71d0dc-95ff-4012-9848-cad7679615c9-kube-api-access-cgwxh\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85\" (UID: \"6a71d0dc-95ff-4012-9848-cad7679615c9\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85" Aug 04 09:47:12.571970 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.571527 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h" Aug 04 09:47:12.635958 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.635926 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x"] Aug 04 09:47:12.673581 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.673550 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85" Aug 04 09:47:12.679061 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:47:12.678569 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2c2d1ca_7214_44c9_9257_bde0bc98a16a.slice/crio-87c876e5c1cb73087236e618b7d522619c4e8fdf4ebc53e8d50fa1d96198b8d8 WatchSource:0}: Error finding container 87c876e5c1cb73087236e618b7d522619c4e8fdf4ebc53e8d50fa1d96198b8d8: Status 404 returned error can't find the container with id 87c876e5c1cb73087236e618b7d522619c4e8fdf4ebc53e8d50fa1d96198b8d8 Aug 04 09:47:12.694930 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.694893 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x" event={"ID":"c2c2d1ca-7214-44c9-9257-bde0bc98a16a","Type":"ContainerStarted","Data":"87c876e5c1cb73087236e618b7d522619c4e8fdf4ebc53e8d50fa1d96198b8d8"} Aug 04 09:47:12.696855 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.696756 2572 generic.go:356] "Generic (PLEG): container finished" podID="a69cb15f-3ab7-4b9d-817b-3e177e319c97" containerID="dde80f31606771a46841f954aa3c0c00f4d5fd4ada67dcfb40ae0c25fc5e3565" exitCode=0 Aug 04 09:47:12.696980 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.696910 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v" event={"ID":"a69cb15f-3ab7-4b9d-817b-3e177e319c97","Type":"ContainerDied","Data":"dde80f31606771a46841f954aa3c0c00f4d5fd4ada67dcfb40ae0c25fc5e3565"} Aug 04 09:47:12.696980 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.696954 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v" event={"ID":"a69cb15f-3ab7-4b9d-817b-3e177e319c97","Type":"ContainerStarted","Data":"77e242021e37d46d156008cead0cf22cef24bf71d4acb9f7a201ed0429a9539c"} Aug 04 09:47:12.751520 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.751480 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h"] Aug 04 09:47:12.752746 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:47:12.752716 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb26ed9f_a18a_412d_9b72_c3514a7b864a.slice/crio-6237c5d78bceaabacaa8676521fc96dacbe7107963aee7c663bd5fc10c6a9400 WatchSource:0}: Error finding container 6237c5d78bceaabacaa8676521fc96dacbe7107963aee7c663bd5fc10c6a9400: Status 404 returned error can't find the container with id 6237c5d78bceaabacaa8676521fc96dacbe7107963aee7c663bd5fc10c6a9400 Aug 04 09:47:12.855282 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:12.855251 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85"] Aug 04 09:47:12.908726 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:47:12.908692 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a71d0dc_95ff_4012_9848_cad7679615c9.slice/crio-114f74bd3785facfe997892c02b970ff996ab89c5e94e52006437cc2858fe586 WatchSource:0}: Error finding container 114f74bd3785facfe997892c02b970ff996ab89c5e94e52006437cc2858fe586: Status 404 returned error can't find the container with id 114f74bd3785facfe997892c02b970ff996ab89c5e94e52006437cc2858fe586 Aug 04 09:47:13.703757 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:13.703715 2572 generic.go:356] "Generic (PLEG): container finished" podID="6a71d0dc-95ff-4012-9848-cad7679615c9" containerID="913cbdea47c546fcf6b9399c8128599a9f1e8f50e270e3d8ab77e2251a3b4a68" exitCode=0 Aug 04 09:47:13.704228 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:13.703819 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85" event={"ID":"6a71d0dc-95ff-4012-9848-cad7679615c9","Type":"ContainerDied","Data":"913cbdea47c546fcf6b9399c8128599a9f1e8f50e270e3d8ab77e2251a3b4a68"} Aug 04 09:47:13.704228 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:13.703902 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85" event={"ID":"6a71d0dc-95ff-4012-9848-cad7679615c9","Type":"ContainerStarted","Data":"114f74bd3785facfe997892c02b970ff996ab89c5e94e52006437cc2858fe586"} Aug 04 09:47:13.705458 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:13.705435 2572 generic.go:356] "Generic (PLEG): container finished" podID="c2c2d1ca-7214-44c9-9257-bde0bc98a16a" containerID="52c11830842b068921ef3a27faea582d9191f976bfe38e1433af48d822170467" exitCode=0 Aug 04 09:47:13.705544 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:13.705518 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x" event={"ID":"c2c2d1ca-7214-44c9-9257-bde0bc98a16a","Type":"ContainerDied","Data":"52c11830842b068921ef3a27faea582d9191f976bfe38e1433af48d822170467"} Aug 04 09:47:13.707483 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:13.707459 2572 generic.go:356] "Generic (PLEG): container finished" podID="a69cb15f-3ab7-4b9d-817b-3e177e319c97" containerID="6873ca75fa9ebe225bfd9a4820fc1f23b273d7428c67ea418015cde8197ebccd" exitCode=0 Aug 04 09:47:13.707573 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:13.707530 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v" event={"ID":"a69cb15f-3ab7-4b9d-817b-3e177e319c97","Type":"ContainerDied","Data":"6873ca75fa9ebe225bfd9a4820fc1f23b273d7428c67ea418015cde8197ebccd"} Aug 04 09:47:13.709212 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:13.709195 2572 generic.go:356] "Generic (PLEG): container finished" podID="fb26ed9f-a18a-412d-9b72-c3514a7b864a" containerID="750f707e985a1b3e6e9f7821ef898bcc0c8ededa1a11aa185324df91fb6d7bdd" exitCode=0 Aug 04 09:47:13.709295 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:13.709256 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h" event={"ID":"fb26ed9f-a18a-412d-9b72-c3514a7b864a","Type":"ContainerDied","Data":"750f707e985a1b3e6e9f7821ef898bcc0c8ededa1a11aa185324df91fb6d7bdd"} Aug 04 09:47:13.709337 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:13.709301 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h" event={"ID":"fb26ed9f-a18a-412d-9b72-c3514a7b864a","Type":"ContainerStarted","Data":"6237c5d78bceaabacaa8676521fc96dacbe7107963aee7c663bd5fc10c6a9400"} Aug 04 09:47:14.716426 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:14.716396 2572 generic.go:356] "Generic (PLEG): container finished" podID="6a71d0dc-95ff-4012-9848-cad7679615c9" containerID="db1107037cd1d7202261c13a64925d0eba99dc619985ec5e81454aaff6b70f63" exitCode=0 Aug 04 09:47:14.716922 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:14.716459 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85" event={"ID":"6a71d0dc-95ff-4012-9848-cad7679615c9","Type":"ContainerDied","Data":"db1107037cd1d7202261c13a64925d0eba99dc619985ec5e81454aaff6b70f63"} Aug 04 09:47:14.718400 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:14.718310 2572 generic.go:356] "Generic (PLEG): container finished" podID="c2c2d1ca-7214-44c9-9257-bde0bc98a16a" containerID="b29a73e0c0208c5412831a46807667e8314c0c6f8c6245669dc7e5c439c38fdf" exitCode=0 Aug 04 09:47:14.718400 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:14.718335 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x" event={"ID":"c2c2d1ca-7214-44c9-9257-bde0bc98a16a","Type":"ContainerDied","Data":"b29a73e0c0208c5412831a46807667e8314c0c6f8c6245669dc7e5c439c38fdf"} Aug 04 09:47:14.720573 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:14.720551 2572 generic.go:356] "Generic (PLEG): container finished" podID="a69cb15f-3ab7-4b9d-817b-3e177e319c97" containerID="6e7e8b101b513acdd856d88907e83c5db4455cc713187113c1943fe6fc6257ea" exitCode=0 Aug 04 09:47:14.720659 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:14.720626 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v" event={"ID":"a69cb15f-3ab7-4b9d-817b-3e177e319c97","Type":"ContainerDied","Data":"6e7e8b101b513acdd856d88907e83c5db4455cc713187113c1943fe6fc6257ea"} Aug 04 09:47:15.727745 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:15.727651 2572 generic.go:356] "Generic (PLEG): container finished" podID="6a71d0dc-95ff-4012-9848-cad7679615c9" containerID="bc612172fc5b9b7cc0285f1608c316b5921bf75cc27d58234aef75eb49587aa5" exitCode=0 Aug 04 09:47:15.728157 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:15.727786 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85" event={"ID":"6a71d0dc-95ff-4012-9848-cad7679615c9","Type":"ContainerDied","Data":"bc612172fc5b9b7cc0285f1608c316b5921bf75cc27d58234aef75eb49587aa5"} Aug 04 09:47:15.729712 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:15.729687 2572 generic.go:356] "Generic (PLEG): container finished" podID="c2c2d1ca-7214-44c9-9257-bde0bc98a16a" containerID="6499fa842564a295bef72948a9abb381c28c9ba3c23d5fe9685501c86cdd64a7" exitCode=0 Aug 04 09:47:15.729792 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:15.729772 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x" event={"ID":"c2c2d1ca-7214-44c9-9257-bde0bc98a16a","Type":"ContainerDied","Data":"6499fa842564a295bef72948a9abb381c28c9ba3c23d5fe9685501c86cdd64a7"} Aug 04 09:47:15.731303 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:15.731282 2572 generic.go:356] "Generic (PLEG): container finished" podID="fb26ed9f-a18a-412d-9b72-c3514a7b864a" containerID="25538b5e830ed9ce695c2e03f2c9b7499a0c5aa42433ff195b02c81ddddf5a52" exitCode=0 Aug 04 09:47:15.731407 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:15.731366 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h" event={"ID":"fb26ed9f-a18a-412d-9b72-c3514a7b864a","Type":"ContainerDied","Data":"25538b5e830ed9ce695c2e03f2c9b7499a0c5aa42433ff195b02c81ddddf5a52"} Aug 04 09:47:15.942909 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:15.942879 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v" Aug 04 09:47:16.046503 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:16.046404 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a69cb15f-3ab7-4b9d-817b-3e177e319c97-bundle\") pod \"a69cb15f-3ab7-4b9d-817b-3e177e319c97\" (UID: \"a69cb15f-3ab7-4b9d-817b-3e177e319c97\") " Aug 04 09:47:16.046503 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:16.046455 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a69cb15f-3ab7-4b9d-817b-3e177e319c97-util\") pod \"a69cb15f-3ab7-4b9d-817b-3e177e319c97\" (UID: \"a69cb15f-3ab7-4b9d-817b-3e177e319c97\") " Aug 04 09:47:16.046503 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:16.046508 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2ff4\" (UniqueName: \"kubernetes.io/projected/a69cb15f-3ab7-4b9d-817b-3e177e319c97-kube-api-access-d2ff4\") pod \"a69cb15f-3ab7-4b9d-817b-3e177e319c97\" (UID: \"a69cb15f-3ab7-4b9d-817b-3e177e319c97\") " Aug 04 09:47:16.047221 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:16.047197 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a69cb15f-3ab7-4b9d-817b-3e177e319c97-bundle" (OuterVolumeSpecName: "bundle") pod "a69cb15f-3ab7-4b9d-817b-3e177e319c97" (UID: "a69cb15f-3ab7-4b9d-817b-3e177e319c97"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:47:16.048727 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:16.048701 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a69cb15f-3ab7-4b9d-817b-3e177e319c97-kube-api-access-d2ff4" (OuterVolumeSpecName: "kube-api-access-d2ff4") pod "a69cb15f-3ab7-4b9d-817b-3e177e319c97" (UID: "a69cb15f-3ab7-4b9d-817b-3e177e319c97"). InnerVolumeSpecName "kube-api-access-d2ff4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:47:16.052524 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:16.052484 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a69cb15f-3ab7-4b9d-817b-3e177e319c97-util" (OuterVolumeSpecName: "util") pod "a69cb15f-3ab7-4b9d-817b-3e177e319c97" (UID: "a69cb15f-3ab7-4b9d-817b-3e177e319c97"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:47:16.148092 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:16.148052 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2ff4\" (UniqueName: \"kubernetes.io/projected/a69cb15f-3ab7-4b9d-817b-3e177e319c97-kube-api-access-d2ff4\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:47:16.148092 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:16.148093 2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a69cb15f-3ab7-4b9d-817b-3e177e319c97-bundle\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:47:16.148092 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:16.148104 2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a69cb15f-3ab7-4b9d-817b-3e177e319c97-util\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:47:16.738496 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:16.738409 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v" event={"ID":"a69cb15f-3ab7-4b9d-817b-3e177e319c97","Type":"ContainerDied","Data":"77e242021e37d46d156008cead0cf22cef24bf71d4acb9f7a201ed0429a9539c"} Aug 04 09:47:16.738496 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:16.738490 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="77e242021e37d46d156008cead0cf22cef24bf71d4acb9f7a201ed0429a9539c" Aug 04 09:47:16.739025 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:16.738425 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053pd55v" Aug 04 09:47:16.740583 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:16.740555 2572 generic.go:356] "Generic (PLEG): container finished" podID="fb26ed9f-a18a-412d-9b72-c3514a7b864a" containerID="7869d16a11f4c8c02350b7dfc30c4622f38696f0938e6e294d0e711be5671d83" exitCode=0 Aug 04 09:47:16.740694 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:16.740626 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h" event={"ID":"fb26ed9f-a18a-412d-9b72-c3514a7b864a","Type":"ContainerDied","Data":"7869d16a11f4c8c02350b7dfc30c4622f38696f0938e6e294d0e711be5671d83"} Aug 04 09:47:16.886296 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:16.886270 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x" Aug 04 09:47:16.914713 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:16.914680 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85" Aug 04 09:47:16.954968 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:16.954933 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2c2d1ca-7214-44c9-9257-bde0bc98a16a-util\") pod \"c2c2d1ca-7214-44c9-9257-bde0bc98a16a\" (UID: \"c2c2d1ca-7214-44c9-9257-bde0bc98a16a\") " Aug 04 09:47:16.955157 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:16.955032 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjxj7\" (UniqueName: \"kubernetes.io/projected/c2c2d1ca-7214-44c9-9257-bde0bc98a16a-kube-api-access-pjxj7\") pod \"c2c2d1ca-7214-44c9-9257-bde0bc98a16a\" (UID: \"c2c2d1ca-7214-44c9-9257-bde0bc98a16a\") " Aug 04 09:47:16.955157 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:16.955061 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2c2d1ca-7214-44c9-9257-bde0bc98a16a-bundle\") pod \"c2c2d1ca-7214-44c9-9257-bde0bc98a16a\" (UID: \"c2c2d1ca-7214-44c9-9257-bde0bc98a16a\") " Aug 04 09:47:16.956226 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:16.956187 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2c2d1ca-7214-44c9-9257-bde0bc98a16a-bundle" (OuterVolumeSpecName: "bundle") pod "c2c2d1ca-7214-44c9-9257-bde0bc98a16a" (UID: "c2c2d1ca-7214-44c9-9257-bde0bc98a16a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:47:16.957390 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:16.957366 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2c2d1ca-7214-44c9-9257-bde0bc98a16a-kube-api-access-pjxj7" (OuterVolumeSpecName: "kube-api-access-pjxj7") pod "c2c2d1ca-7214-44c9-9257-bde0bc98a16a" (UID: "c2c2d1ca-7214-44c9-9257-bde0bc98a16a"). InnerVolumeSpecName "kube-api-access-pjxj7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:47:16.961902 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:16.961864 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2c2d1ca-7214-44c9-9257-bde0bc98a16a-util" (OuterVolumeSpecName: "util") pod "c2c2d1ca-7214-44c9-9257-bde0bc98a16a" (UID: "c2c2d1ca-7214-44c9-9257-bde0bc98a16a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:47:17.055972 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:17.055873 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgwxh\" (UniqueName: \"kubernetes.io/projected/6a71d0dc-95ff-4012-9848-cad7679615c9-kube-api-access-cgwxh\") pod \"6a71d0dc-95ff-4012-9848-cad7679615c9\" (UID: \"6a71d0dc-95ff-4012-9848-cad7679615c9\") " Aug 04 09:47:17.055972 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:17.055951 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a71d0dc-95ff-4012-9848-cad7679615c9-util\") pod \"6a71d0dc-95ff-4012-9848-cad7679615c9\" (UID: \"6a71d0dc-95ff-4012-9848-cad7679615c9\") " Aug 04 09:47:17.056206 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:17.055992 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a71d0dc-95ff-4012-9848-cad7679615c9-bundle\") pod \"6a71d0dc-95ff-4012-9848-cad7679615c9\" (UID: \"6a71d0dc-95ff-4012-9848-cad7679615c9\") " Aug 04 09:47:17.056206 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:17.056198 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjxj7\" (UniqueName: \"kubernetes.io/projected/c2c2d1ca-7214-44c9-9257-bde0bc98a16a-kube-api-access-pjxj7\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:47:17.056538 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:17.056208 2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2c2d1ca-7214-44c9-9257-bde0bc98a16a-bundle\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:47:17.056538 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:17.056217 2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2c2d1ca-7214-44c9-9257-bde0bc98a16a-util\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:47:17.056710 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:17.056530 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a71d0dc-95ff-4012-9848-cad7679615c9-bundle" (OuterVolumeSpecName: "bundle") pod "6a71d0dc-95ff-4012-9848-cad7679615c9" (UID: "6a71d0dc-95ff-4012-9848-cad7679615c9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:47:17.058077 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:17.058050 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a71d0dc-95ff-4012-9848-cad7679615c9-kube-api-access-cgwxh" (OuterVolumeSpecName: "kube-api-access-cgwxh") pod "6a71d0dc-95ff-4012-9848-cad7679615c9" (UID: "6a71d0dc-95ff-4012-9848-cad7679615c9"). InnerVolumeSpecName "kube-api-access-cgwxh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:47:17.065086 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:17.065049 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a71d0dc-95ff-4012-9848-cad7679615c9-util" (OuterVolumeSpecName: "util") pod "6a71d0dc-95ff-4012-9848-cad7679615c9" (UID: "6a71d0dc-95ff-4012-9848-cad7679615c9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:47:17.157214 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:17.157173 2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a71d0dc-95ff-4012-9848-cad7679615c9-bundle\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:47:17.157214 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:17.157208 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgwxh\" (UniqueName: \"kubernetes.io/projected/6a71d0dc-95ff-4012-9848-cad7679615c9-kube-api-access-cgwxh\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:47:17.157417 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:17.157233 2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a71d0dc-95ff-4012-9848-cad7679615c9-util\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:47:17.754876 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:17.754758 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85" event={"ID":"6a71d0dc-95ff-4012-9848-cad7679615c9","Type":"ContainerDied","Data":"114f74bd3785facfe997892c02b970ff996ab89c5e94e52006437cc2858fe586"} Aug 04 09:47:17.754876 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:17.754824 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbwgd85" Aug 04 09:47:17.754876 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:17.754873 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="114f74bd3785facfe997892c02b970ff996ab89c5e94e52006437cc2858fe586" Aug 04 09:47:17.756807 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:17.756774 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x" Aug 04 09:47:17.756954 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:17.756792 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c9662x" event={"ID":"c2c2d1ca-7214-44c9-9257-bde0bc98a16a","Type":"ContainerDied","Data":"87c876e5c1cb73087236e618b7d522619c4e8fdf4ebc53e8d50fa1d96198b8d8"} Aug 04 09:47:17.756954 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:17.756850 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="87c876e5c1cb73087236e618b7d522619c4e8fdf4ebc53e8d50fa1d96198b8d8" Aug 04 09:47:17.900711 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:17.900682 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h" Aug 04 09:47:17.964587 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:17.964552 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4wx4\" (UniqueName: \"kubernetes.io/projected/fb26ed9f-a18a-412d-9b72-c3514a7b864a-kube-api-access-j4wx4\") pod \"fb26ed9f-a18a-412d-9b72-c3514a7b864a\" (UID: \"fb26ed9f-a18a-412d-9b72-c3514a7b864a\") " Aug 04 09:47:17.964814 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:17.964639 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb26ed9f-a18a-412d-9b72-c3514a7b864a-bundle\") pod \"fb26ed9f-a18a-412d-9b72-c3514a7b864a\" (UID: \"fb26ed9f-a18a-412d-9b72-c3514a7b864a\") " Aug 04 09:47:17.964814 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:17.964752 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb26ed9f-a18a-412d-9b72-c3514a7b864a-util\") pod \"fb26ed9f-a18a-412d-9b72-c3514a7b864a\" (UID: \"fb26ed9f-a18a-412d-9b72-c3514a7b864a\") " Aug 04 09:47:17.965106 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:17.965085 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb26ed9f-a18a-412d-9b72-c3514a7b864a-bundle" (OuterVolumeSpecName: "bundle") pod "fb26ed9f-a18a-412d-9b72-c3514a7b864a" (UID: "fb26ed9f-a18a-412d-9b72-c3514a7b864a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:47:17.966725 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:17.966699 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb26ed9f-a18a-412d-9b72-c3514a7b864a-kube-api-access-j4wx4" (OuterVolumeSpecName: "kube-api-access-j4wx4") pod "fb26ed9f-a18a-412d-9b72-c3514a7b864a" (UID: "fb26ed9f-a18a-412d-9b72-c3514a7b864a"). InnerVolumeSpecName "kube-api-access-j4wx4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:47:18.014070 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:18.013955 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb26ed9f-a18a-412d-9b72-c3514a7b864a-util" (OuterVolumeSpecName: "util") pod "fb26ed9f-a18a-412d-9b72-c3514a7b864a" (UID: "fb26ed9f-a18a-412d-9b72-c3514a7b864a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:47:18.065620 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:18.065584 2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb26ed9f-a18a-412d-9b72-c3514a7b864a-util\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:47:18.065620 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:18.065616 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4wx4\" (UniqueName: \"kubernetes.io/projected/fb26ed9f-a18a-412d-9b72-c3514a7b864a-kube-api-access-j4wx4\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:47:18.065620 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:18.065627 2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb26ed9f-a18a-412d-9b72-c3514a7b864a-bundle\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:47:18.763359 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:18.763327 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h" Aug 04 09:47:18.763745 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:18.763331 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b589h" event={"ID":"fb26ed9f-a18a-412d-9b72-c3514a7b864a","Type":"ContainerDied","Data":"6237c5d78bceaabacaa8676521fc96dacbe7107963aee7c663bd5fc10c6a9400"} Aug 04 09:47:18.763745 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:18.763443 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6237c5d78bceaabacaa8676521fc96dacbe7107963aee7c663bd5fc10c6a9400" Aug 04 09:47:23.758281 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:23.758246 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:25.913085 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.913049 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8tdd5"] Aug 04 09:47:25.913455 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.913386 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a69cb15f-3ab7-4b9d-817b-3e177e319c97" containerName="extract" Aug 04 09:47:25.913455 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.913397 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a69cb15f-3ab7-4b9d-817b-3e177e319c97" containerName="extract" Aug 04 09:47:25.913455 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.913406 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a69cb15f-3ab7-4b9d-817b-3e177e319c97" containerName="util" Aug 04 09:47:25.913455 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.913412 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a69cb15f-3ab7-4b9d-817b-3e177e319c97" containerName="util" Aug 04 09:47:25.913455 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.913421 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2c2d1ca-7214-44c9-9257-bde0bc98a16a" containerName="util" Aug 04 09:47:25.913455 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.913426 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2c2d1ca-7214-44c9-9257-bde0bc98a16a" containerName="util" Aug 04 09:47:25.913455 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.913434 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb26ed9f-a18a-412d-9b72-c3514a7b864a" containerName="extract" Aug 04 09:47:25.913455 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.913439 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb26ed9f-a18a-412d-9b72-c3514a7b864a" containerName="extract" Aug 04 09:47:25.913455 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.913448 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a71d0dc-95ff-4012-9848-cad7679615c9" containerName="util" Aug 04 09:47:25.913455 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.913453 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a71d0dc-95ff-4012-9848-cad7679615c9" containerName="util" Aug 04 09:47:25.913455 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.913459 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a71d0dc-95ff-4012-9848-cad7679615c9" containerName="pull" Aug 04 09:47:25.913856 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.913465 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a71d0dc-95ff-4012-9848-cad7679615c9" containerName="pull" Aug 04 09:47:25.913856 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.913477 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2c2d1ca-7214-44c9-9257-bde0bc98a16a" containerName="pull" Aug 04 09:47:25.913856 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.913482 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2c2d1ca-7214-44c9-9257-bde0bc98a16a" containerName="pull" Aug 04 09:47:25.913856 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.913490 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb26ed9f-a18a-412d-9b72-c3514a7b864a" containerName="util" Aug 04 09:47:25.913856 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.913495 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb26ed9f-a18a-412d-9b72-c3514a7b864a" containerName="util" Aug 04 09:47:25.913856 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.913500 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb26ed9f-a18a-412d-9b72-c3514a7b864a" containerName="pull" Aug 04 09:47:25.913856 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.913504 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb26ed9f-a18a-412d-9b72-c3514a7b864a" containerName="pull" Aug 04 09:47:25.913856 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.913512 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a71d0dc-95ff-4012-9848-cad7679615c9" containerName="extract" Aug 04 09:47:25.913856 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.913516 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a71d0dc-95ff-4012-9848-cad7679615c9" containerName="extract" Aug 04 09:47:25.913856 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.913525 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a69cb15f-3ab7-4b9d-817b-3e177e319c97" containerName="pull" Aug 04 09:47:25.913856 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.913530 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a69cb15f-3ab7-4b9d-817b-3e177e319c97" containerName="pull" Aug 04 09:47:25.913856 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.913537 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2c2d1ca-7214-44c9-9257-bde0bc98a16a" containerName="extract" Aug 04 09:47:25.913856 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.913542 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2c2d1ca-7214-44c9-9257-bde0bc98a16a" containerName="extract" Aug 04 09:47:25.913856 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.913593 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a71d0dc-95ff-4012-9848-cad7679615c9" containerName="extract" Aug 04 09:47:25.913856 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.913602 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a69cb15f-3ab7-4b9d-817b-3e177e319c97" containerName="extract" Aug 04 09:47:25.913856 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.913608 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb26ed9f-a18a-412d-9b72-c3514a7b864a" containerName="extract" Aug 04 09:47:25.913856 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.913615 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2c2d1ca-7214-44c9-9257-bde0bc98a16a" containerName="extract" Aug 04 09:47:25.916597 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.916581 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8tdd5" Aug 04 09:47:25.921017 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.920998 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Aug 04 09:47:25.921593 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.921576 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-8bxmf\"" Aug 04 09:47:25.921666 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.921583 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Aug 04 09:47:25.931491 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:25.931463 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8tdd5"] Aug 04 09:47:26.035861 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:26.035788 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22sdr\" (UniqueName: \"kubernetes.io/projected/b618e1cc-73b3-4029-af39-1018b11583b3-kube-api-access-22sdr\") pod \"limitador-operator-controller-manager-6c549b6d85-8tdd5\" (UID: \"b618e1cc-73b3-4029-af39-1018b11583b3\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8tdd5" Aug 04 09:47:26.136415 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:26.136373 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22sdr\" (UniqueName: \"kubernetes.io/projected/b618e1cc-73b3-4029-af39-1018b11583b3-kube-api-access-22sdr\") pod \"limitador-operator-controller-manager-6c549b6d85-8tdd5\" (UID: \"b618e1cc-73b3-4029-af39-1018b11583b3\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8tdd5" Aug 04 09:47:26.164970 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:26.164899 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22sdr\" (UniqueName: \"kubernetes.io/projected/b618e1cc-73b3-4029-af39-1018b11583b3-kube-api-access-22sdr\") pod \"limitador-operator-controller-manager-6c549b6d85-8tdd5\" (UID: \"b618e1cc-73b3-4029-af39-1018b11583b3\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8tdd5" Aug 04 09:47:26.227343 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:26.227309 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8tdd5" Aug 04 09:47:26.392659 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:26.392631 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8tdd5"] Aug 04 09:47:26.596360 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:26.596314 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:26.806630 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:26.806593 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8tdd5" event={"ID":"b618e1cc-73b3-4029-af39-1018b11583b3","Type":"ContainerStarted","Data":"4cbf038b6ed29d6dcfcebf85cc34656095220c0b6a976cd3adf2e52f2ad38ef3"} Aug 04 09:47:26.961169 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:26.961138 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:27.383978 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:27.383838 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:28.415652 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:28.415619 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-9d7445797-6m6t7"] Aug 04 09:47:28.419416 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:28.419384 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9d7445797-6m6t7" Aug 04 09:47:28.432545 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:28.432516 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9d7445797-6m6t7"] Aug 04 09:47:28.526219 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:28.526146 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:28.561620 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:28.561582 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc-console-config\") pod \"console-9d7445797-6m6t7\" (UID: \"3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc\") " pod="openshift-console/console-9d7445797-6m6t7" Aug 04 09:47:28.561852 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:28.561644 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc-service-ca\") pod \"console-9d7445797-6m6t7\" (UID: \"3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc\") " pod="openshift-console/console-9d7445797-6m6t7" Aug 04 09:47:28.561852 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:28.561670 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8sf5\" (UniqueName: \"kubernetes.io/projected/3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc-kube-api-access-w8sf5\") pod \"console-9d7445797-6m6t7\" (UID: \"3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc\") " pod="openshift-console/console-9d7445797-6m6t7" Aug 04 09:47:28.561852 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:28.561780 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc-console-oauth-config\") pod \"console-9d7445797-6m6t7\" (UID: \"3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc\") " pod="openshift-console/console-9d7445797-6m6t7" Aug 04 09:47:28.562044 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:28.561935 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc-trusted-ca-bundle\") pod \"console-9d7445797-6m6t7\" (UID: \"3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc\") " pod="openshift-console/console-9d7445797-6m6t7" Aug 04 09:47:28.562044 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:28.561988 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc-console-serving-cert\") pod \"console-9d7445797-6m6t7\" (UID: \"3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc\") " pod="openshift-console/console-9d7445797-6m6t7" Aug 04 09:47:28.562044 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:28.562018 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc-oauth-serving-cert\") pod \"console-9d7445797-6m6t7\" (UID: \"3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc\") " pod="openshift-console/console-9d7445797-6m6t7" Aug 04 09:47:28.663066 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:28.663022 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc-trusted-ca-bundle\") pod \"console-9d7445797-6m6t7\" (UID: \"3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc\") " pod="openshift-console/console-9d7445797-6m6t7" Aug 04 09:47:28.663271 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:28.663089 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc-console-serving-cert\") pod \"console-9d7445797-6m6t7\" (UID: \"3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc\") " pod="openshift-console/console-9d7445797-6m6t7" Aug 04 09:47:28.663271 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:28.663119 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc-oauth-serving-cert\") pod \"console-9d7445797-6m6t7\" (UID: \"3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc\") " pod="openshift-console/console-9d7445797-6m6t7" Aug 04 09:47:28.663271 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:28.663151 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc-console-config\") pod \"console-9d7445797-6m6t7\" (UID: \"3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc\") " pod="openshift-console/console-9d7445797-6m6t7" Aug 04 09:47:28.663271 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:28.663189 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc-service-ca\") pod \"console-9d7445797-6m6t7\" (UID: \"3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc\") " pod="openshift-console/console-9d7445797-6m6t7" Aug 04 09:47:28.663271 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:28.663211 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8sf5\" (UniqueName: \"kubernetes.io/projected/3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc-kube-api-access-w8sf5\") pod \"console-9d7445797-6m6t7\" (UID: \"3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc\") " pod="openshift-console/console-9d7445797-6m6t7" Aug 04 09:47:28.663543 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:28.663346 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc-console-oauth-config\") pod \"console-9d7445797-6m6t7\" (UID: \"3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc\") " pod="openshift-console/console-9d7445797-6m6t7" Aug 04 09:47:28.664067 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:28.664037 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc-trusted-ca-bundle\") pod \"console-9d7445797-6m6t7\" (UID: \"3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc\") " pod="openshift-console/console-9d7445797-6m6t7" Aug 04 09:47:28.664266 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:28.664238 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc-console-config\") pod \"console-9d7445797-6m6t7\" (UID: \"3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc\") " pod="openshift-console/console-9d7445797-6m6t7" Aug 04 09:47:28.664392 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:28.664238 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc-service-ca\") pod \"console-9d7445797-6m6t7\" (UID: \"3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc\") " pod="openshift-console/console-9d7445797-6m6t7" Aug 04 09:47:28.664392 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:28.664285 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc-oauth-serving-cert\") pod \"console-9d7445797-6m6t7\" (UID: \"3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc\") " pod="openshift-console/console-9d7445797-6m6t7" Aug 04 09:47:28.666500 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:28.666434 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc-console-oauth-config\") pod \"console-9d7445797-6m6t7\" (UID: \"3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc\") " pod="openshift-console/console-9d7445797-6m6t7" Aug 04 09:47:28.666500 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:28.666440 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc-console-serving-cert\") pod \"console-9d7445797-6m6t7\" (UID: \"3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc\") " pod="openshift-console/console-9d7445797-6m6t7" Aug 04 09:47:28.677615 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:28.677582 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:28.681918 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:28.681890 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8sf5\" (UniqueName: \"kubernetes.io/projected/3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc-kube-api-access-w8sf5\") pod \"console-9d7445797-6m6t7\" (UID: \"3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc\") " pod="openshift-console/console-9d7445797-6m6t7" Aug 04 09:47:28.731335 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:28.731299 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9d7445797-6m6t7" Aug 04 09:47:28.924314 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:28.924243 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9d7445797-6m6t7"] Aug 04 09:47:28.926911 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:47:28.926887 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3099ba6e_e9c2_4a88_a4ae_00b4a12ee7fc.slice/crio-5409b95cf7fdb863d6e9a6ea568a4ea09ca0fdcab042f30da4244a6050c06402 WatchSource:0}: Error finding container 5409b95cf7fdb863d6e9a6ea568a4ea09ca0fdcab042f30da4244a6050c06402: Status 404 returned error can't find the container with id 5409b95cf7fdb863d6e9a6ea568a4ea09ca0fdcab042f30da4244a6050c06402 Aug 04 09:47:29.230529 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:29.230495 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:29.825185 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:29.825146 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8tdd5" event={"ID":"b618e1cc-73b3-4029-af39-1018b11583b3","Type":"ContainerStarted","Data":"8edb7fedf1a80c83bb2669cee9949685787f50ff55a9ec3ef9cb5488e7d31671"} Aug 04 09:47:29.825639 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:29.825255 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8tdd5" Aug 04 09:47:29.826728 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:29.826704 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9d7445797-6m6t7" event={"ID":"3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc","Type":"ContainerStarted","Data":"b26b95930c605f122be8faffcd855efa0ea323d619314c0bb46c023f701af0e2"} Aug 04 09:47:29.826855 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:29.826740 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9d7445797-6m6t7" event={"ID":"3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc","Type":"ContainerStarted","Data":"5409b95cf7fdb863d6e9a6ea568a4ea09ca0fdcab042f30da4244a6050c06402"} Aug 04 09:47:29.848366 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:29.848316 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8tdd5" podStartSLOduration=2.460280779 podStartE2EDuration="4.84829777s" totalImagesPullingTime="2.388016991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:47:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:47:29.848237291 +0000 UTC m=+418.592371710" watchObservedRunningTime="2026-08-04 09:47:29.84829777 +0000 UTC m=+418.592432190" Aug 04 09:47:29.877463 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:29.877414 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-9d7445797-6m6t7" podStartSLOduration=1.877399464 podStartE2EDuration="1.877399464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:47:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:47:29.876627756 +0000 UTC m=+418.620762167" watchObservedRunningTime="2026-08-04 09:47:29.877399464 +0000 UTC m=+418.621533884" Aug 04 09:47:30.384534 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:30.384501 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:30.626556 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:30.626520 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:31.053662 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:31.053628 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5f6g7"] Aug 04 09:47:31.062037 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:31.062002 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5f6g7" Aug 04 09:47:31.065040 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:31.065014 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-wp5hm\"" Aug 04 09:47:31.079155 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:31.079122 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5f6g7"] Aug 04 09:47:31.178529 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:31.178496 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:31.187752 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:31.187713 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/556c5a94-827a-4993-b97e-1905416cc129-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-5f6g7\" (UID: \"556c5a94-827a-4993-b97e-1905416cc129\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5f6g7" Aug 04 09:47:31.187887 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:31.187776 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9c68\" (UniqueName: \"kubernetes.io/projected/556c5a94-827a-4993-b97e-1905416cc129-kube-api-access-t9c68\") pod \"kuadrant-operator-controller-manager-58fc56757c-5f6g7\" (UID: \"556c5a94-827a-4993-b97e-1905416cc129\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5f6g7" Aug 04 09:47:31.288534 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:31.288490 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/556c5a94-827a-4993-b97e-1905416cc129-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-5f6g7\" (UID: \"556c5a94-827a-4993-b97e-1905416cc129\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5f6g7" Aug 04 09:47:31.288687 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:31.288556 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9c68\" (UniqueName: \"kubernetes.io/projected/556c5a94-827a-4993-b97e-1905416cc129-kube-api-access-t9c68\") pod \"kuadrant-operator-controller-manager-58fc56757c-5f6g7\" (UID: \"556c5a94-827a-4993-b97e-1905416cc129\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5f6g7" Aug 04 09:47:31.288927 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:31.288909 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/556c5a94-827a-4993-b97e-1905416cc129-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-5f6g7\" (UID: \"556c5a94-827a-4993-b97e-1905416cc129\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5f6g7" Aug 04 09:47:31.303604 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:31.303576 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9c68\" (UniqueName: \"kubernetes.io/projected/556c5a94-827a-4993-b97e-1905416cc129-kube-api-access-t9c68\") pod \"kuadrant-operator-controller-manager-58fc56757c-5f6g7\" (UID: \"556c5a94-827a-4993-b97e-1905416cc129\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5f6g7" Aug 04 09:47:31.372540 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:31.372449 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5f6g7" Aug 04 09:47:31.531008 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:31.530976 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5f6g7"] Aug 04 09:47:31.531476 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:47:31.531444 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod556c5a94_827a_4993_b97e_1905416cc129.slice/crio-005b2ffacaa5ea16e75ac90c476735c6a79df17e8d57ede73b481642bcfc5777 WatchSource:0}: Error finding container 005b2ffacaa5ea16e75ac90c476735c6a79df17e8d57ede73b481642bcfc5777: Status 404 returned error can't find the container with id 005b2ffacaa5ea16e75ac90c476735c6a79df17e8d57ede73b481642bcfc5777 Aug 04 09:47:31.838963 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:31.838926 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5f6g7" event={"ID":"556c5a94-827a-4993-b97e-1905416cc129","Type":"ContainerStarted","Data":"005b2ffacaa5ea16e75ac90c476735c6a79df17e8d57ede73b481642bcfc5777"} Aug 04 09:47:32.656436 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:32.656389 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:32.683240 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:32.683199 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-89kk7"] Aug 04 09:47:32.686729 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:32.686705 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-89kk7" Aug 04 09:47:32.691256 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:32.691227 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-n9szc\"" Aug 04 09:47:32.691757 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:32.691740 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Aug 04 09:47:32.708917 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:32.708884 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-89kk7"] Aug 04 09:47:32.784463 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:32.784429 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:32.802956 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:32.802923 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqqnz\" (UniqueName: \"kubernetes.io/projected/79788882-8745-4be1-9ede-95fa3bf45a3d-kube-api-access-mqqnz\") pod \"dns-operator-controller-manager-5985564776-89kk7\" (UID: \"79788882-8745-4be1-9ede-95fa3bf45a3d\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-89kk7" Aug 04 09:47:32.904662 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:32.904621 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqqnz\" (UniqueName: \"kubernetes.io/projected/79788882-8745-4be1-9ede-95fa3bf45a3d-kube-api-access-mqqnz\") pod \"dns-operator-controller-manager-5985564776-89kk7\" (UID: \"79788882-8745-4be1-9ede-95fa3bf45a3d\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-89kk7" Aug 04 09:47:32.919756 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:32.919679 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqqnz\" (UniqueName: \"kubernetes.io/projected/79788882-8745-4be1-9ede-95fa3bf45a3d-kube-api-access-mqqnz\") pod \"dns-operator-controller-manager-5985564776-89kk7\" (UID: \"79788882-8745-4be1-9ede-95fa3bf45a3d\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-89kk7" Aug 04 09:47:33.000359 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:33.000326 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-89kk7" Aug 04 09:47:33.186049 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:33.185904 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-89kk7"] Aug 04 09:47:33.189824 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:47:33.189773 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79788882_8745_4be1_9ede_95fa3bf45a3d.slice/crio-3d9782fa05c3350f50ac5b376acd28b12b448103fb7e6b719fcc1de1e94b2773 WatchSource:0}: Error finding container 3d9782fa05c3350f50ac5b376acd28b12b448103fb7e6b719fcc1de1e94b2773: Status 404 returned error can't find the container with id 3d9782fa05c3350f50ac5b376acd28b12b448103fb7e6b719fcc1de1e94b2773 Aug 04 09:47:33.335304 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:33.335266 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:33.853258 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:33.853214 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-89kk7" event={"ID":"79788882-8745-4be1-9ede-95fa3bf45a3d","Type":"ContainerStarted","Data":"3d9782fa05c3350f50ac5b376acd28b12b448103fb7e6b719fcc1de1e94b2773"} Aug 04 09:47:35.868466 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:35.868413 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5f6g7" event={"ID":"556c5a94-827a-4993-b97e-1905416cc129","Type":"ContainerStarted","Data":"d2a69e1569d6bf1632c3af5a08680490ad94cf304ecc371d4ec927a3ce8ba33a"} Aug 04 09:47:35.869049 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:35.868506 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5f6g7" Aug 04 09:47:35.941408 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:35.941351 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5f6g7" podStartSLOduration=1.090383631 podStartE2EDuration="4.941335303s" totalImagesPullingTime="3.850951672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:47:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:47:35.921304395 +0000 UTC m=+424.665438817" watchObservedRunningTime="2026-08-04 09:47:35.941335303 +0000 UTC m=+424.685469722" Aug 04 09:47:36.684943 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:36.684846 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-7shjd"] Aug 04 09:47:36.688509 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:36.688489 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-7shjd" Aug 04 09:47:36.693890 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:36.693865 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-rgt5h\"" Aug 04 09:47:36.706612 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:36.706577 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-7shjd"] Aug 04 09:47:36.845301 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:36.845262 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88h29\" (UniqueName: \"kubernetes.io/projected/29bf9ef7-f611-4f5b-a7d2-f4054e519d00-kube-api-access-88h29\") pod \"authorino-operator-68d88948d8-7shjd\" (UID: \"29bf9ef7-f611-4f5b-a7d2-f4054e519d00\") " pod="kuadrant-system/authorino-operator-68d88948d8-7shjd" Aug 04 09:47:36.875851 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:36.875785 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-89kk7" event={"ID":"79788882-8745-4be1-9ede-95fa3bf45a3d","Type":"ContainerStarted","Data":"a43c9bd8e5237159ea6a4d3c84b91ca561004d1631387ff3d9324dc1fea9d863"} Aug 04 09:47:36.876270 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:36.876074 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-89kk7" Aug 04 09:47:36.908809 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:36.908753 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-5985564776-89kk7" podStartSLOduration=1.831369983 podStartE2EDuration="4.908735452s" totalImagesPullingTime="3.077365469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:47:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:47:36.90729884 +0000 UTC m=+425.651433260" watchObservedRunningTime="2026-08-04 09:47:36.908735452 +0000 UTC m=+425.652869870" Aug 04 09:47:36.945908 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:36.945872 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88h29\" (UniqueName: \"kubernetes.io/projected/29bf9ef7-f611-4f5b-a7d2-f4054e519d00-kube-api-access-88h29\") pod \"authorino-operator-68d88948d8-7shjd\" (UID: \"29bf9ef7-f611-4f5b-a7d2-f4054e519d00\") " pod="kuadrant-system/authorino-operator-68d88948d8-7shjd" Aug 04 09:47:36.964121 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:36.964082 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88h29\" (UniqueName: \"kubernetes.io/projected/29bf9ef7-f611-4f5b-a7d2-f4054e519d00-kube-api-access-88h29\") pod \"authorino-operator-68d88948d8-7shjd\" (UID: \"29bf9ef7-f611-4f5b-a7d2-f4054e519d00\") " pod="kuadrant-system/authorino-operator-68d88948d8-7shjd" Aug 04 09:47:36.999222 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:36.999179 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-7shjd" Aug 04 09:47:37.176166 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:37.176140 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-7shjd"] Aug 04 09:47:37.178380 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:47:37.178355 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29bf9ef7_f611_4f5b_a7d2_f4054e519d00.slice/crio-d1dbdea1e8f6079c2051ab20d63af2a46d369b7351fdd64732cf1688a9c8604d WatchSource:0}: Error finding container d1dbdea1e8f6079c2051ab20d63af2a46d369b7351fdd64732cf1688a9c8604d: Status 404 returned error can't find the container with id d1dbdea1e8f6079c2051ab20d63af2a46d369b7351fdd64732cf1688a9c8604d Aug 04 09:47:37.882885 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:37.882843 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-7shjd" event={"ID":"29bf9ef7-f611-4f5b-a7d2-f4054e519d00","Type":"ContainerStarted","Data":"d1dbdea1e8f6079c2051ab20d63af2a46d369b7351fdd64732cf1688a9c8604d"} Aug 04 09:47:38.159928 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:38.159835 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:38.286156 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:38.286119 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:38.582295 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:38.582256 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-9c4tp"] Aug 04 09:47:38.585829 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:38.585805 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-9c4tp" Aug 04 09:47:38.593840 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:38.593817 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-878ss\"" Aug 04 09:47:38.593994 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:38.593877 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Aug 04 09:47:38.607967 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:38.607934 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Aug 04 09:47:38.610453 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:38.610417 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-9c4tp"] Aug 04 09:47:38.622662 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:38.622619 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:38.732300 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:38.732259 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9d7445797-6m6t7" Aug 04 09:47:38.732496 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:38.732326 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-9d7445797-6m6t7" Aug 04 09:47:38.738223 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:38.738190 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-9d7445797-6m6t7" Aug 04 09:47:38.764202 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:38.764164 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b8b566f6-96fd-493e-ad07-1eb86d960581-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-9c4tp\" (UID: \"b8b566f6-96fd-493e-ad07-1eb86d960581\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-9c4tp" Aug 04 09:47:38.764367 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:38.764219 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5psxt\" (UniqueName: \"kubernetes.io/projected/b8b566f6-96fd-493e-ad07-1eb86d960581-kube-api-access-5psxt\") pod \"kuadrant-console-plugin-598f86468b-9c4tp\" (UID: \"b8b566f6-96fd-493e-ad07-1eb86d960581\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-9c4tp" Aug 04 09:47:38.764416 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:38.764371 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8b566f6-96fd-493e-ad07-1eb86d960581-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-9c4tp\" (UID: \"b8b566f6-96fd-493e-ad07-1eb86d960581\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-9c4tp" Aug 04 09:47:38.779240 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:38.778663 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:38.866047 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:38.865962 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8b566f6-96fd-493e-ad07-1eb86d960581-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-9c4tp\" (UID: \"b8b566f6-96fd-493e-ad07-1eb86d960581\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-9c4tp" Aug 04 09:47:38.866255 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:38.866227 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b8b566f6-96fd-493e-ad07-1eb86d960581-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-9c4tp\" (UID: \"b8b566f6-96fd-493e-ad07-1eb86d960581\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-9c4tp" Aug 04 09:47:38.866727 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:38.866623 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5psxt\" (UniqueName: \"kubernetes.io/projected/b8b566f6-96fd-493e-ad07-1eb86d960581-kube-api-access-5psxt\") pod \"kuadrant-console-plugin-598f86468b-9c4tp\" (UID: \"b8b566f6-96fd-493e-ad07-1eb86d960581\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-9c4tp" Aug 04 09:47:38.872401 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:38.868039 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b8b566f6-96fd-493e-ad07-1eb86d960581-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-9c4tp\" (UID: \"b8b566f6-96fd-493e-ad07-1eb86d960581\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-9c4tp" Aug 04 09:47:38.872401 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:38.870244 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8b566f6-96fd-493e-ad07-1eb86d960581-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-9c4tp\" (UID: \"b8b566f6-96fd-493e-ad07-1eb86d960581\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-9c4tp" Aug 04 09:47:38.882481 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:38.882447 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5psxt\" (UniqueName: \"kubernetes.io/projected/b8b566f6-96fd-493e-ad07-1eb86d960581-kube-api-access-5psxt\") pod \"kuadrant-console-plugin-598f86468b-9c4tp\" (UID: \"b8b566f6-96fd-493e-ad07-1eb86d960581\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-9c4tp" Aug 04 09:47:38.892040 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:38.892013 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-9d7445797-6m6t7" Aug 04 09:47:38.897066 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:38.897043 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-9c4tp" Aug 04 09:47:38.975633 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:38.975541 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-689b989dbb-xjpfk"] Aug 04 09:47:39.095224 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:47:39.095192 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb8b566f6_96fd_493e_ad07_1eb86d960581.slice/crio-72e8cf7c057d580529ed3639f59d868cbb4aa9ba9c16f652b2b55eea0fa37652 WatchSource:0}: Error finding container 72e8cf7c057d580529ed3639f59d868cbb4aa9ba9c16f652b2b55eea0fa37652: Status 404 returned error can't find the container with id 72e8cf7c057d580529ed3639f59d868cbb4aa9ba9c16f652b2b55eea0fa37652 Aug 04 09:47:39.095524 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:39.095502 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-9c4tp"] Aug 04 09:47:39.168333 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:39.168234 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:39.329744 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:39.329707 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:39.439401 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:39.439319 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:39.895138 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:39.895105 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-7shjd" event={"ID":"29bf9ef7-f611-4f5b-a7d2-f4054e519d00","Type":"ContainerStarted","Data":"e3258c059b8535118066ab28f204311683fbfa6cf83d037d6cd68b399c44b5fb"} Aug 04 09:47:39.895550 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:39.895216 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d88948d8-7shjd" Aug 04 09:47:39.896463 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:39.896437 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-9c4tp" event={"ID":"b8b566f6-96fd-493e-ad07-1eb86d960581","Type":"ContainerStarted","Data":"72e8cf7c057d580529ed3639f59d868cbb4aa9ba9c16f652b2b55eea0fa37652"} Aug 04 09:47:39.930756 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:39.930699 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d88948d8-7shjd" podStartSLOduration=2.199451126 podStartE2EDuration="3.930682743s" totalImagesPullingTime="1.731231617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:47:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:47:39.928932727 +0000 UTC m=+428.673067147" watchObservedRunningTime="2026-08-04 09:47:39.930682743 +0000 UTC m=+428.674817161" Aug 04 09:47:40.254877 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:40.254745 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:40.555505 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:40.555409 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:40.835510 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:40.835424 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8tdd5" Aug 04 09:47:42.590955 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:42.590920 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:42.715396 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:42.715355 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:42.814056 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:42.813923 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:44.939230 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:44.939192 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-9c4tp" event={"ID":"b8b566f6-96fd-493e-ad07-1eb86d960581","Type":"ContainerStarted","Data":"c2739158b67ecc5fe044522deb415722abdeaab16dd21822fbbc0e379c1154ae"} Aug 04 09:47:44.966904 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:44.966843 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-9c4tp" podStartSLOduration=2.145633926 podStartE2EDuration="6.966824194s" totalImagesPullingTime="4.821190268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:47:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:47:44.963667943 +0000 UTC m=+433.707802361" watchObservedRunningTime="2026-08-04 09:47:44.966824194 +0000 UTC m=+433.710958614" Aug 04 09:47:46.879097 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:46.879063 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5f6g7" Aug 04 09:47:47.885887 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:47.885844 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-89kk7" Aug 04 09:47:49.149947 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.149911 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5f6g7"] Aug 04 09:47:49.150352 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.150089 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5f6g7" podUID="556c5a94-827a-4993-b97e-1905416cc129" containerName="manager" containerID="cri-o://d2a69e1569d6bf1632c3af5a08680490ad94cf304ecc371d4ec927a3ce8ba33a" gracePeriod=2 Aug 04 09:47:49.164718 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.164683 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5f6g7"] Aug 04 09:47:49.198660 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.198619 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8tdd5"] Aug 04 09:47:49.199233 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.199199 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8tdd5" podUID="b618e1cc-73b3-4029-af39-1018b11583b3" containerName="manager" containerID="cri-o://8edb7fedf1a80c83bb2669cee9949685787f50ff55a9ec3ef9cb5488e7d31671" gracePeriod=2 Aug 04 09:47:49.239353 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.239168 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8tdd5"] Aug 04 09:47:49.270906 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:47:49.270858 2572 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"b618e1cc-73b3-4029-af39-1018b11583b3\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-6c549b6d85-8tdd5\": pods \"limitador-operator-controller-manager-6c549b6d85-8tdd5\" not found" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8tdd5" Aug 04 09:47:49.273306 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.273276 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fjlxt"] Aug 04 09:47:49.273715 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.273699 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="556c5a94-827a-4993-b97e-1905416cc129" containerName="manager" Aug 04 09:47:49.273793 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.273719 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="556c5a94-827a-4993-b97e-1905416cc129" containerName="manager" Aug 04 09:47:49.273793 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.273733 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b618e1cc-73b3-4029-af39-1018b11583b3" containerName="manager" Aug 04 09:47:49.273793 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.273741 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b618e1cc-73b3-4029-af39-1018b11583b3" containerName="manager" Aug 04 09:47:49.273975 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.273840 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="556c5a94-827a-4993-b97e-1905416cc129" containerName="manager" Aug 04 09:47:49.273975 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.273858 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b618e1cc-73b3-4029-af39-1018b11583b3" containerName="manager" Aug 04 09:47:49.276969 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.276949 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fjlxt" Aug 04 09:47:49.283077 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:47:49.283047 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-5f6g7\" is forbidden: User \"system:node:ip-10-0-129-241.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-241.ec2.internal' and this object" podUID="556c5a94-827a-4993-b97e-1905416cc129" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5f6g7" Aug 04 09:47:49.343391 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.343308 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fjlxt"] Aug 04 09:47:49.354554 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.354511 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mxgw\" (UniqueName: \"kubernetes.io/projected/e75bf807-5958-4c94-8f34-69611f539dc8-kube-api-access-5mxgw\") pod \"limitador-operator-controller-manager-6c549b6d85-fjlxt\" (UID: \"e75bf807-5958-4c94-8f34-69611f539dc8\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fjlxt" Aug 04 09:47:49.456161 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.456121 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mxgw\" (UniqueName: \"kubernetes.io/projected/e75bf807-5958-4c94-8f34-69611f539dc8-kube-api-access-5mxgw\") pod \"limitador-operator-controller-manager-6c549b6d85-fjlxt\" (UID: \"e75bf807-5958-4c94-8f34-69611f539dc8\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fjlxt" Aug 04 09:47:49.460076 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:47:49.460030 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-8tdd5\" is forbidden: User \"system:node:ip-10-0-129-241.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-241.ec2.internal' and this object" podUID="b618e1cc-73b3-4029-af39-1018b11583b3" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8tdd5" Aug 04 09:47:49.465107 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:47:49.465067 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-5f6g7\" is forbidden: User \"system:node:ip-10-0-129-241.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-241.ec2.internal' and this object" podUID="556c5a94-827a-4993-b97e-1905416cc129" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5f6g7" Aug 04 09:47:49.482053 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.482020 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mxgw\" (UniqueName: \"kubernetes.io/projected/e75bf807-5958-4c94-8f34-69611f539dc8-kube-api-access-5mxgw\") pod \"limitador-operator-controller-manager-6c549b6d85-fjlxt\" (UID: \"e75bf807-5958-4c94-8f34-69611f539dc8\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fjlxt" Aug 04 09:47:49.489478 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.489452 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8tdd5" Aug 04 09:47:49.492768 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:47:49.492738 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-8tdd5\" is forbidden: User \"system:node:ip-10-0-129-241.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-241.ec2.internal' and this object" podUID="b618e1cc-73b3-4029-af39-1018b11583b3" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8tdd5" Aug 04 09:47:49.493448 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.493432 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5f6g7" Aug 04 09:47:49.495353 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:47:49.495326 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-5f6g7\" is forbidden: User \"system:node:ip-10-0-129-241.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-241.ec2.internal' and this object" podUID="556c5a94-827a-4993-b97e-1905416cc129" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5f6g7" Aug 04 09:47:49.499540 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:47:49.499515 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-8tdd5\" is forbidden: User \"system:node:ip-10-0-129-241.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-241.ec2.internal' and this object" podUID="b618e1cc-73b3-4029-af39-1018b11583b3" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8tdd5" Aug 04 09:47:49.501981 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:47:49.501950 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-5f6g7\" is forbidden: User \"system:node:ip-10-0-129-241.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-241.ec2.internal' and this object" podUID="556c5a94-827a-4993-b97e-1905416cc129" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5f6g7" Aug 04 09:47:49.557306 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.557265 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22sdr\" (UniqueName: \"kubernetes.io/projected/b618e1cc-73b3-4029-af39-1018b11583b3-kube-api-access-22sdr\") pod \"b618e1cc-73b3-4029-af39-1018b11583b3\" (UID: \"b618e1cc-73b3-4029-af39-1018b11583b3\") " Aug 04 09:47:49.557306 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.557307 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9c68\" (UniqueName: \"kubernetes.io/projected/556c5a94-827a-4993-b97e-1905416cc129-kube-api-access-t9c68\") pod \"556c5a94-827a-4993-b97e-1905416cc129\" (UID: \"556c5a94-827a-4993-b97e-1905416cc129\") " Aug 04 09:47:49.557559 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.557376 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/556c5a94-827a-4993-b97e-1905416cc129-extensions-socket-volume\") pod \"556c5a94-827a-4993-b97e-1905416cc129\" (UID: \"556c5a94-827a-4993-b97e-1905416cc129\") " Aug 04 09:47:49.557869 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.557841 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/556c5a94-827a-4993-b97e-1905416cc129-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "556c5a94-827a-4993-b97e-1905416cc129" (UID: "556c5a94-827a-4993-b97e-1905416cc129"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 09:47:49.559566 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.559542 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b618e1cc-73b3-4029-af39-1018b11583b3-kube-api-access-22sdr" (OuterVolumeSpecName: "kube-api-access-22sdr") pod "b618e1cc-73b3-4029-af39-1018b11583b3" (UID: "b618e1cc-73b3-4029-af39-1018b11583b3"). InnerVolumeSpecName "kube-api-access-22sdr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:47:49.559629 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.559565 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/556c5a94-827a-4993-b97e-1905416cc129-kube-api-access-t9c68" (OuterVolumeSpecName: "kube-api-access-t9c68") pod "556c5a94-827a-4993-b97e-1905416cc129" (UID: "556c5a94-827a-4993-b97e-1905416cc129"). InnerVolumeSpecName "kube-api-access-t9c68". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:47:49.596986 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.596946 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fjlxt" Aug 04 09:47:49.658960 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.658869 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22sdr\" (UniqueName: \"kubernetes.io/projected/b618e1cc-73b3-4029-af39-1018b11583b3-kube-api-access-22sdr\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:47:49.658960 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.658907 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9c68\" (UniqueName: \"kubernetes.io/projected/556c5a94-827a-4993-b97e-1905416cc129-kube-api-access-t9c68\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:47:49.658960 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.658923 2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/556c5a94-827a-4993-b97e-1905416cc129-extensions-socket-volume\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:47:49.675209 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.675175 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:49.760929 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.760901 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fjlxt"] Aug 04 09:47:49.761770 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:47:49.761746 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode75bf807_5958_4c94_8f34_69611f539dc8.slice/crio-90ba0754266f8326d7f199db21cb0a3286275822eaff7c6beceafdfa7299522a WatchSource:0}: Error finding container 90ba0754266f8326d7f199db21cb0a3286275822eaff7c6beceafdfa7299522a: Status 404 returned error can't find the container with id 90ba0754266f8326d7f199db21cb0a3286275822eaff7c6beceafdfa7299522a Aug 04 09:47:49.781226 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.781202 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="556c5a94-827a-4993-b97e-1905416cc129" path="/var/lib/kubelet/pods/556c5a94-827a-4993-b97e-1905416cc129/volumes" Aug 04 09:47:49.781531 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.781519 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b618e1cc-73b3-4029-af39-1018b11583b3" path="/var/lib/kubelet/pods/b618e1cc-73b3-4029-af39-1018b11583b3/volumes" Aug 04 09:47:49.837029 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.836992 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:49.968461 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.968423 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fjlxt" event={"ID":"e75bf807-5958-4c94-8f34-69611f539dc8","Type":"ContainerStarted","Data":"043f3593443832d81830daca5e0718fe3517071f621b2409811c139cbbb07352"} Aug 04 09:47:49.968671 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.968494 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fjlxt" event={"ID":"e75bf807-5958-4c94-8f34-69611f539dc8","Type":"ContainerStarted","Data":"90ba0754266f8326d7f199db21cb0a3286275822eaff7c6beceafdfa7299522a"} Aug 04 09:47:49.968671 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.968620 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fjlxt" Aug 04 09:47:49.969872 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.969845 2572 generic.go:356] "Generic (PLEG): container finished" podID="b618e1cc-73b3-4029-af39-1018b11583b3" containerID="8edb7fedf1a80c83bb2669cee9949685787f50ff55a9ec3ef9cb5488e7d31671" exitCode=0 Aug 04 09:47:49.970002 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.969877 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-8tdd5" Aug 04 09:47:49.970002 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.969940 2572 scope.go:117] "RemoveContainer" containerID="8edb7fedf1a80c83bb2669cee9949685787f50ff55a9ec3ef9cb5488e7d31671" Aug 04 09:47:49.971447 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.971424 2572 generic.go:356] "Generic (PLEG): container finished" podID="556c5a94-827a-4993-b97e-1905416cc129" containerID="d2a69e1569d6bf1632c3af5a08680490ad94cf304ecc371d4ec927a3ce8ba33a" exitCode=0 Aug 04 09:47:49.971551 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.971482 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5f6g7" Aug 04 09:47:49.979056 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.979034 2572 scope.go:117] "RemoveContainer" containerID="8edb7fedf1a80c83bb2669cee9949685787f50ff55a9ec3ef9cb5488e7d31671" Aug 04 09:47:49.979414 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:47:49.979390 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8edb7fedf1a80c83bb2669cee9949685787f50ff55a9ec3ef9cb5488e7d31671\": container with ID starting with 8edb7fedf1a80c83bb2669cee9949685787f50ff55a9ec3ef9cb5488e7d31671 not found: ID does not exist" containerID="8edb7fedf1a80c83bb2669cee9949685787f50ff55a9ec3ef9cb5488e7d31671" Aug 04 09:47:49.979482 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.979424 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8edb7fedf1a80c83bb2669cee9949685787f50ff55a9ec3ef9cb5488e7d31671"} err="failed to get container status \"8edb7fedf1a80c83bb2669cee9949685787f50ff55a9ec3ef9cb5488e7d31671\": rpc error: code = NotFound desc = could not find container \"8edb7fedf1a80c83bb2669cee9949685787f50ff55a9ec3ef9cb5488e7d31671\": container with ID starting with 8edb7fedf1a80c83bb2669cee9949685787f50ff55a9ec3ef9cb5488e7d31671 not found: ID does not exist" Aug 04 09:47:49.979482 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.979443 2572 scope.go:117] "RemoveContainer" containerID="d2a69e1569d6bf1632c3af5a08680490ad94cf304ecc371d4ec927a3ce8ba33a" Aug 04 09:47:49.988402 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.988381 2572 scope.go:117] "RemoveContainer" containerID="d2a69e1569d6bf1632c3af5a08680490ad94cf304ecc371d4ec927a3ce8ba33a" Aug 04 09:47:49.988735 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:47:49.988715 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2a69e1569d6bf1632c3af5a08680490ad94cf304ecc371d4ec927a3ce8ba33a\": container with ID starting with d2a69e1569d6bf1632c3af5a08680490ad94cf304ecc371d4ec927a3ce8ba33a not found: ID does not exist" containerID="d2a69e1569d6bf1632c3af5a08680490ad94cf304ecc371d4ec927a3ce8ba33a" Aug 04 09:47:49.988785 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:49.988744 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2a69e1569d6bf1632c3af5a08680490ad94cf304ecc371d4ec927a3ce8ba33a"} err="failed to get container status \"d2a69e1569d6bf1632c3af5a08680490ad94cf304ecc371d4ec927a3ce8ba33a\": rpc error: code = NotFound desc = could not find container \"d2a69e1569d6bf1632c3af5a08680490ad94cf304ecc371d4ec927a3ce8ba33a\": container with ID starting with d2a69e1569d6bf1632c3af5a08680490ad94cf304ecc371d4ec927a3ce8ba33a not found: ID does not exist" Aug 04 09:47:50.004594 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:50.004541 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fjlxt" podStartSLOduration=1.004524811 podStartE2EDuration="1.004524811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:47:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:47:50.002624974 +0000 UTC m=+438.746759392" watchObservedRunningTime="2026-08-04 09:47:50.004524811 +0000 UTC m=+438.748659229" Aug 04 09:47:50.907502 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:50.907474 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d88948d8-7shjd" Aug 04 09:47:51.826931 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:51.826893 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lp98f"] Aug 04 09:47:51.831781 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:51.831757 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lp98f" Aug 04 09:47:51.839525 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:51.839502 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-wp5hm\"" Aug 04 09:47:51.854543 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:51.854510 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lp98f"] Aug 04 09:47:51.878050 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:51.878015 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfgnd\" (UniqueName: \"kubernetes.io/projected/210a1804-8468-4310-9779-e7c9a3d3eeb7-kube-api-access-wfgnd\") pod \"kuadrant-operator-controller-manager-68cf589589-lp98f\" (UID: \"210a1804-8468-4310-9779-e7c9a3d3eeb7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lp98f" Aug 04 09:47:51.878226 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:51.878086 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/210a1804-8468-4310-9779-e7c9a3d3eeb7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-lp98f\" (UID: \"210a1804-8468-4310-9779-e7c9a3d3eeb7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lp98f" Aug 04 09:47:51.979159 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:51.979121 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfgnd\" (UniqueName: \"kubernetes.io/projected/210a1804-8468-4310-9779-e7c9a3d3eeb7-kube-api-access-wfgnd\") pod \"kuadrant-operator-controller-manager-68cf589589-lp98f\" (UID: \"210a1804-8468-4310-9779-e7c9a3d3eeb7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lp98f" Aug 04 09:47:51.979634 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:51.979183 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/210a1804-8468-4310-9779-e7c9a3d3eeb7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-lp98f\" (UID: \"210a1804-8468-4310-9779-e7c9a3d3eeb7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lp98f" Aug 04 09:47:51.979634 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:51.979515 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/210a1804-8468-4310-9779-e7c9a3d3eeb7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-lp98f\" (UID: \"210a1804-8468-4310-9779-e7c9a3d3eeb7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lp98f" Aug 04 09:47:52.002030 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:52.001993 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfgnd\" (UniqueName: \"kubernetes.io/projected/210a1804-8468-4310-9779-e7c9a3d3eeb7-kube-api-access-wfgnd\") pod \"kuadrant-operator-controller-manager-68cf589589-lp98f\" (UID: \"210a1804-8468-4310-9779-e7c9a3d3eeb7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lp98f" Aug 04 09:47:52.142308 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:52.142206 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lp98f" Aug 04 09:47:52.317110 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:52.317081 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lp98f"] Aug 04 09:47:52.319176 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:47:52.319130 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod210a1804_8468_4310_9779_e7c9a3d3eeb7.slice/crio-024edd440f6c9c078f217c80190b1e103ccea24d24f6dfa26ed04670e931bbcf WatchSource:0}: Error finding container 024edd440f6c9c078f217c80190b1e103ccea24d24f6dfa26ed04670e931bbcf: Status 404 returned error can't find the container with id 024edd440f6c9c078f217c80190b1e103ccea24d24f6dfa26ed04670e931bbcf Aug 04 09:47:52.992173 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:52.992132 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lp98f" event={"ID":"210a1804-8468-4310-9779-e7c9a3d3eeb7","Type":"ContainerStarted","Data":"82339c8473d646f6f37fdbcafaaa1e682aafb6e1aae989d22c8c4a6e062406c1"} Aug 04 09:47:52.992590 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:52.992230 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lp98f" event={"ID":"210a1804-8468-4310-9779-e7c9a3d3eeb7","Type":"ContainerStarted","Data":"024edd440f6c9c078f217c80190b1e103ccea24d24f6dfa26ed04670e931bbcf"} Aug 04 09:47:52.992590 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:52.992260 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lp98f" Aug 04 09:47:53.020875 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:53.020793 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lp98f" podStartSLOduration=2.020771195 podStartE2EDuration="2.020771195s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:47:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:47:53.01914805 +0000 UTC m=+441.763282474" watchObservedRunningTime="2026-08-04 09:47:53.020771195 +0000 UTC m=+441.764905614" Aug 04 09:47:55.774365 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:55.774329 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:55.891340 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:55.891301 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:58.590930 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:58.590877 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:47:58.723027 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:47:58.722993 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:00.982342 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:00.982310 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fjlxt" Aug 04 09:48:01.919412 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:01.919375 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:02.045988 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:02.045940 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:02.171388 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:02.171290 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:04.001090 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:04.001059 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lp98f" Aug 04 09:48:04.009758 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:04.009724 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-689b989dbb-xjpfk" podUID="5a805a34-24d3-4538-81a4-65aed5e764dc" containerName="console" containerID="cri-o://4aa09d498515c0ff4950b2bf87c5f66235c899f552e5fe9fd3ae7db738f08fca" gracePeriod=15 Aug 04 09:48:04.274222 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:04.274196 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-689b989dbb-xjpfk" Aug 04 09:48:04.395200 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:04.395171 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bp9ch\" (UniqueName: \"kubernetes.io/projected/5a805a34-24d3-4538-81a4-65aed5e764dc-kube-api-access-bp9ch\") pod \"5a805a34-24d3-4538-81a4-65aed5e764dc\" (UID: \"5a805a34-24d3-4538-81a4-65aed5e764dc\") " Aug 04 09:48:04.395356 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:04.395209 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a805a34-24d3-4538-81a4-65aed5e764dc-service-ca\") pod \"5a805a34-24d3-4538-81a4-65aed5e764dc\" (UID: \"5a805a34-24d3-4538-81a4-65aed5e764dc\") " Aug 04 09:48:04.395356 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:04.395241 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a805a34-24d3-4538-81a4-65aed5e764dc-console-oauth-config\") pod \"5a805a34-24d3-4538-81a4-65aed5e764dc\" (UID: \"5a805a34-24d3-4538-81a4-65aed5e764dc\") " Aug 04 09:48:04.395356 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:04.395308 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a805a34-24d3-4538-81a4-65aed5e764dc-console-config\") pod \"5a805a34-24d3-4538-81a4-65aed5e764dc\" (UID: \"5a805a34-24d3-4538-81a4-65aed5e764dc\") " Aug 04 09:48:04.395487 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:04.395363 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a805a34-24d3-4538-81a4-65aed5e764dc-trusted-ca-bundle\") pod \"5a805a34-24d3-4538-81a4-65aed5e764dc\" (UID: \"5a805a34-24d3-4538-81a4-65aed5e764dc\") " Aug 04 09:48:04.395487 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:04.395403 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a805a34-24d3-4538-81a4-65aed5e764dc-oauth-serving-cert\") pod \"5a805a34-24d3-4538-81a4-65aed5e764dc\" (UID: \"5a805a34-24d3-4538-81a4-65aed5e764dc\") " Aug 04 09:48:04.395487 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:04.395436 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a805a34-24d3-4538-81a4-65aed5e764dc-console-serving-cert\") pod \"5a805a34-24d3-4538-81a4-65aed5e764dc\" (UID: \"5a805a34-24d3-4538-81a4-65aed5e764dc\") " Aug 04 09:48:04.396201 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:04.396163 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a805a34-24d3-4538-81a4-65aed5e764dc-service-ca" (OuterVolumeSpecName: "service-ca") pod "5a805a34-24d3-4538-81a4-65aed5e764dc" (UID: "5a805a34-24d3-4538-81a4-65aed5e764dc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:48:04.396424 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:04.396199 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a805a34-24d3-4538-81a4-65aed5e764dc-console-config" (OuterVolumeSpecName: "console-config") pod "5a805a34-24d3-4538-81a4-65aed5e764dc" (UID: "5a805a34-24d3-4538-81a4-65aed5e764dc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:48:04.396526 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:04.396506 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a805a34-24d3-4538-81a4-65aed5e764dc-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5a805a34-24d3-4538-81a4-65aed5e764dc" (UID: "5a805a34-24d3-4538-81a4-65aed5e764dc"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:48:04.397071 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:04.397052 2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a805a34-24d3-4538-81a4-65aed5e764dc-service-ca\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:48:04.397142 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:04.397086 2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a805a34-24d3-4538-81a4-65aed5e764dc-console-config\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:48:04.397142 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:04.397100 2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a805a34-24d3-4538-81a4-65aed5e764dc-trusted-ca-bundle\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:48:04.398176 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:04.398142 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a805a34-24d3-4538-81a4-65aed5e764dc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5a805a34-24d3-4538-81a4-65aed5e764dc" (UID: "5a805a34-24d3-4538-81a4-65aed5e764dc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 09:48:04.406671 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:04.406608 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a805a34-24d3-4538-81a4-65aed5e764dc-kube-api-access-bp9ch" (OuterVolumeSpecName: "kube-api-access-bp9ch") pod "5a805a34-24d3-4538-81a4-65aed5e764dc" (UID: "5a805a34-24d3-4538-81a4-65aed5e764dc"). InnerVolumeSpecName "kube-api-access-bp9ch". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:48:04.406784 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:04.406755 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a805a34-24d3-4538-81a4-65aed5e764dc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5a805a34-24d3-4538-81a4-65aed5e764dc" (UID: "5a805a34-24d3-4538-81a4-65aed5e764dc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:48:04.406784 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:04.406770 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a805a34-24d3-4538-81a4-65aed5e764dc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5a805a34-24d3-4538-81a4-65aed5e764dc" (UID: "5a805a34-24d3-4538-81a4-65aed5e764dc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 09:48:04.498502 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:04.498460 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bp9ch\" (UniqueName: \"kubernetes.io/projected/5a805a34-24d3-4538-81a4-65aed5e764dc-kube-api-access-bp9ch\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:48:04.498502 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:04.498494 2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a805a34-24d3-4538-81a4-65aed5e764dc-console-oauth-config\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:48:04.498502 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:04.498503 2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a805a34-24d3-4538-81a4-65aed5e764dc-oauth-serving-cert\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:48:04.498502 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:04.498512 2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a805a34-24d3-4538-81a4-65aed5e764dc-console-serving-cert\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:48:05.062759 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:05.062725 2572 generic.go:356] "Generic (PLEG): container finished" podID="5a805a34-24d3-4538-81a4-65aed5e764dc" containerID="4aa09d498515c0ff4950b2bf87c5f66235c899f552e5fe9fd3ae7db738f08fca" exitCode=0 Aug 04 09:48:05.063183 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:05.062784 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-689b989dbb-xjpfk" Aug 04 09:48:05.063183 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:05.062827 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-689b989dbb-xjpfk" event={"ID":"5a805a34-24d3-4538-81a4-65aed5e764dc","Type":"ContainerDied","Data":"4aa09d498515c0ff4950b2bf87c5f66235c899f552e5fe9fd3ae7db738f08fca"} Aug 04 09:48:05.063183 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:05.062883 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-689b989dbb-xjpfk" event={"ID":"5a805a34-24d3-4538-81a4-65aed5e764dc","Type":"ContainerDied","Data":"bab2e65c6612a7e5885fa2164c05e61b760b2512c7580076ea19cc3400cf4d97"} Aug 04 09:48:05.063183 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:05.062904 2572 scope.go:117] "RemoveContainer" containerID="4aa09d498515c0ff4950b2bf87c5f66235c899f552e5fe9fd3ae7db738f08fca" Aug 04 09:48:05.072674 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:05.072652 2572 scope.go:117] "RemoveContainer" containerID="4aa09d498515c0ff4950b2bf87c5f66235c899f552e5fe9fd3ae7db738f08fca" Aug 04 09:48:05.073028 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:48:05.073009 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4aa09d498515c0ff4950b2bf87c5f66235c899f552e5fe9fd3ae7db738f08fca\": container with ID starting with 4aa09d498515c0ff4950b2bf87c5f66235c899f552e5fe9fd3ae7db738f08fca not found: ID does not exist" containerID="4aa09d498515c0ff4950b2bf87c5f66235c899f552e5fe9fd3ae7db738f08fca" Aug 04 09:48:05.073111 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:05.073048 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4aa09d498515c0ff4950b2bf87c5f66235c899f552e5fe9fd3ae7db738f08fca"} err="failed to get container status \"4aa09d498515c0ff4950b2bf87c5f66235c899f552e5fe9fd3ae7db738f08fca\": rpc error: code = NotFound desc = could not find container \"4aa09d498515c0ff4950b2bf87c5f66235c899f552e5fe9fd3ae7db738f08fca\": container with ID starting with 4aa09d498515c0ff4950b2bf87c5f66235c899f552e5fe9fd3ae7db738f08fca not found: ID does not exist" Aug 04 09:48:05.092180 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:05.092154 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-689b989dbb-xjpfk"] Aug 04 09:48:05.094401 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:05.094377 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-689b989dbb-xjpfk"] Aug 04 09:48:05.778896 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:05.778857 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a805a34-24d3-4538-81a4-65aed5e764dc" path="/var/lib/kubelet/pods/5a805a34-24d3-4538-81a4-65aed5e764dc/volumes" Aug 04 09:48:08.322648 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:08.322595 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:08.406305 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:08.406271 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:14.854422 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:14.854384 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:14.954558 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:14.954525 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:17.596707 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:17.596671 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:17.695088 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:17.695046 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:20.697312 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.697273 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q"] Aug 04 09:48:20.697891 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.697875 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a805a34-24d3-4538-81a4-65aed5e764dc" containerName="console" Aug 04 09:48:20.697958 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.697895 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a805a34-24d3-4538-81a4-65aed5e764dc" containerName="console" Aug 04 09:48:20.698014 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.698003 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a805a34-24d3-4538-81a4-65aed5e764dc" containerName="console" Aug 04 09:48:20.719583 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.719549 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q"] Aug 04 09:48:20.719782 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.719624 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:20.723059 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.723032 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-vvfj5\"" Aug 04 09:48:20.723964 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.723940 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:20.848773 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.848728 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c84a000-d9ad-433f-881b-5cb6e5b32088-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6284q\" (UID: \"1c84a000-d9ad-433f-881b-5cb6e5b32088\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:20.849013 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.848782 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c84a000-d9ad-433f-881b-5cb6e5b32088-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6284q\" (UID: \"1c84a000-d9ad-433f-881b-5cb6e5b32088\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:20.849013 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.848866 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c84a000-d9ad-433f-881b-5cb6e5b32088-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6284q\" (UID: \"1c84a000-d9ad-433f-881b-5cb6e5b32088\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:20.849013 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.848900 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c84a000-d9ad-433f-881b-5cb6e5b32088-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6284q\" (UID: \"1c84a000-d9ad-433f-881b-5cb6e5b32088\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:20.849013 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.848932 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c84a000-d9ad-433f-881b-5cb6e5b32088-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6284q\" (UID: \"1c84a000-d9ad-433f-881b-5cb6e5b32088\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:20.849183 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.849026 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c84a000-d9ad-433f-881b-5cb6e5b32088-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6284q\" (UID: \"1c84a000-d9ad-433f-881b-5cb6e5b32088\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:20.849183 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.849073 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c84a000-d9ad-433f-881b-5cb6e5b32088-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6284q\" (UID: \"1c84a000-d9ad-433f-881b-5cb6e5b32088\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:20.849183 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.849099 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfb5z\" (UniqueName: \"kubernetes.io/projected/1c84a000-d9ad-433f-881b-5cb6e5b32088-kube-api-access-nfb5z\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6284q\" (UID: \"1c84a000-d9ad-433f-881b-5cb6e5b32088\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:20.849183 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.849123 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c84a000-d9ad-433f-881b-5cb6e5b32088-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6284q\" (UID: \"1c84a000-d9ad-433f-881b-5cb6e5b32088\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:20.950205 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.950106 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c84a000-d9ad-433f-881b-5cb6e5b32088-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6284q\" (UID: \"1c84a000-d9ad-433f-881b-5cb6e5b32088\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:20.950205 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.950154 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c84a000-d9ad-433f-881b-5cb6e5b32088-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6284q\" (UID: \"1c84a000-d9ad-433f-881b-5cb6e5b32088\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:20.950409 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.950384 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c84a000-d9ad-433f-881b-5cb6e5b32088-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6284q\" (UID: \"1c84a000-d9ad-433f-881b-5cb6e5b32088\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:20.950460 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.950447 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c84a000-d9ad-433f-881b-5cb6e5b32088-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6284q\" (UID: \"1c84a000-d9ad-433f-881b-5cb6e5b32088\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:20.950499 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.950487 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfb5z\" (UniqueName: \"kubernetes.io/projected/1c84a000-d9ad-433f-881b-5cb6e5b32088-kube-api-access-nfb5z\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6284q\" (UID: \"1c84a000-d9ad-433f-881b-5cb6e5b32088\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:20.950564 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.950522 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c84a000-d9ad-433f-881b-5cb6e5b32088-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6284q\" (UID: \"1c84a000-d9ad-433f-881b-5cb6e5b32088\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:20.950622 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.950558 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c84a000-d9ad-433f-881b-5cb6e5b32088-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6284q\" (UID: \"1c84a000-d9ad-433f-881b-5cb6e5b32088\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:20.950679 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.950640 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c84a000-d9ad-433f-881b-5cb6e5b32088-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6284q\" (UID: \"1c84a000-d9ad-433f-881b-5cb6e5b32088\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:20.950736 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.950698 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c84a000-d9ad-433f-881b-5cb6e5b32088-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6284q\" (UID: \"1c84a000-d9ad-433f-881b-5cb6e5b32088\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:20.950792 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.950754 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c84a000-d9ad-433f-881b-5cb6e5b32088-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6284q\" (UID: \"1c84a000-d9ad-433f-881b-5cb6e5b32088\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:20.950894 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.950873 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c84a000-d9ad-433f-881b-5cb6e5b32088-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6284q\" (UID: \"1c84a000-d9ad-433f-881b-5cb6e5b32088\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:20.951127 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.951101 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c84a000-d9ad-433f-881b-5cb6e5b32088-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6284q\" (UID: \"1c84a000-d9ad-433f-881b-5cb6e5b32088\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:20.951127 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.951119 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c84a000-d9ad-433f-881b-5cb6e5b32088-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6284q\" (UID: \"1c84a000-d9ad-433f-881b-5cb6e5b32088\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:20.951579 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.951561 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c84a000-d9ad-433f-881b-5cb6e5b32088-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6284q\" (UID: \"1c84a000-d9ad-433f-881b-5cb6e5b32088\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:20.952859 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.952828 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c84a000-d9ad-433f-881b-5cb6e5b32088-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6284q\" (UID: \"1c84a000-d9ad-433f-881b-5cb6e5b32088\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:20.953495 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.953471 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c84a000-d9ad-433f-881b-5cb6e5b32088-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6284q\" (UID: \"1c84a000-d9ad-433f-881b-5cb6e5b32088\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:20.961003 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.960961 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfb5z\" (UniqueName: \"kubernetes.io/projected/1c84a000-d9ad-433f-881b-5cb6e5b32088-kube-api-access-nfb5z\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6284q\" (UID: \"1c84a000-d9ad-433f-881b-5cb6e5b32088\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:20.961133 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:20.961009 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c84a000-d9ad-433f-881b-5cb6e5b32088-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-6284q\" (UID: \"1c84a000-d9ad-433f-881b-5cb6e5b32088\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:21.031071 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:21.031037 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:21.078079 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:21.078038 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:21.208203 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:21.208154 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q"] Aug 04 09:48:21.208994 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:48:21.208960 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c84a000_d9ad_433f_881b_5cb6e5b32088.slice/crio-eb364008cc5bb2deaa0c1367fbaf15fa282a8e0a0843ae69ca79ba53e6a3ac3c WatchSource:0}: Error finding container eb364008cc5bb2deaa0c1367fbaf15fa282a8e0a0843ae69ca79ba53e6a3ac3c: Status 404 returned error can't find the container with id eb364008cc5bb2deaa0c1367fbaf15fa282a8e0a0843ae69ca79ba53e6a3ac3c Aug 04 09:48:21.211176 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:21.211128 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 09:48:21.211275 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:21.211221 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 09:48:21.211275 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:21.211251 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 09:48:21.219848 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:21.219824 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:21.396495 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:21.396459 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:21.809679 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:21.809649 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:21.899226 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:21.899187 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:22.163624 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:22.163528 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" event={"ID":"1c84a000-d9ad-433f-881b-5cb6e5b32088","Type":"ContainerStarted","Data":"ca194bc984854baab518da779d9f1bc54c214ae89ab19d3005734a0ce287a4a8"} Aug 04 09:48:22.163624 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:22.163591 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" event={"ID":"1c84a000-d9ad-433f-881b-5cb6e5b32088","Type":"ContainerStarted","Data":"eb364008cc5bb2deaa0c1367fbaf15fa282a8e0a0843ae69ca79ba53e6a3ac3c"} Aug 04 09:48:22.223420 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:22.223363 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" podStartSLOduration=2.223344349 podStartE2EDuration="2.223344349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:48:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:48:22.220359048 +0000 UTC m=+470.964493467" watchObservedRunningTime="2026-08-04 09:48:22.223344349 +0000 UTC m=+470.967478767" Aug 04 09:48:23.032009 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:23.031968 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:23.037067 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:23.037040 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:23.169087 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:23.169050 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:23.170194 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:23.170174 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:48:23.376759 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:23.376674 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:23.474630 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:23.474600 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:23.575424 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:23.575364 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:24.293978 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:24.293938 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b54df94b4-6cc2b"] Aug 04 09:48:24.297848 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:24.297789 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-6cc2b" Aug 04 09:48:24.301027 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:24.301004 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-mg4mx\"" Aug 04 09:48:24.317381 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:24.317354 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-6cc2b"] Aug 04 09:48:24.340837 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:24.340773 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:24.441658 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:24.441622 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:24.482931 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:24.482897 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ln88v\" (UniqueName: \"kubernetes.io/projected/b0286333-8b99-45ca-b848-1191e518c690-kube-api-access-ln88v\") pod \"authorino-b54df94b4-6cc2b\" (UID: \"b0286333-8b99-45ca-b848-1191e518c690\") " pod="kuadrant-system/authorino-b54df94b4-6cc2b" Aug 04 09:48:24.584154 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:24.584059 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ln88v\" (UniqueName: \"kubernetes.io/projected/b0286333-8b99-45ca-b848-1191e518c690-kube-api-access-ln88v\") pod \"authorino-b54df94b4-6cc2b\" (UID: \"b0286333-8b99-45ca-b848-1191e518c690\") " pod="kuadrant-system/authorino-b54df94b4-6cc2b" Aug 04 09:48:24.603217 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:24.603187 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ln88v\" (UniqueName: \"kubernetes.io/projected/b0286333-8b99-45ca-b848-1191e518c690-kube-api-access-ln88v\") pod \"authorino-b54df94b4-6cc2b\" (UID: \"b0286333-8b99-45ca-b848-1191e518c690\") " pod="kuadrant-system/authorino-b54df94b4-6cc2b" Aug 04 09:48:24.608033 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:24.608001 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-6cc2b" Aug 04 09:48:24.761950 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:24.761922 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-6cc2b"] Aug 04 09:48:24.764363 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:48:24.764323 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0286333_8b99_45ca_b848_1191e518c690.slice/crio-2b5ad5f47506db046903f9225ac73e7310c77ac4c60db3821538a6f1c6cbd94d WatchSource:0}: Error finding container 2b5ad5f47506db046903f9225ac73e7310c77ac4c60db3821538a6f1c6cbd94d: Status 404 returned error can't find the container with id 2b5ad5f47506db046903f9225ac73e7310c77ac4c60db3821538a6f1c6cbd94d Aug 04 09:48:25.181376 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:25.181337 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-6cc2b" event={"ID":"b0286333-8b99-45ca-b848-1191e518c690","Type":"ContainerStarted","Data":"2b5ad5f47506db046903f9225ac73e7310c77ac4c60db3821538a6f1c6cbd94d"} Aug 04 09:48:27.195085 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:27.195047 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-6cc2b" event={"ID":"b0286333-8b99-45ca-b848-1191e518c690","Type":"ContainerStarted","Data":"60161897b5ed4002912d358ca4b2249f20bf5e24b591aa6b5b6d9fb9210c9b17"} Aug 04 09:48:27.214216 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:27.214152 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b54df94b4-6cc2b" podStartSLOduration=1.448291032 podStartE2EDuration="3.214133501s" totalImagesPullingTime="1.765842469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:48:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:48:27.211513108 +0000 UTC m=+475.955647527" watchObservedRunningTime="2026-08-04 09:48:27.214133501 +0000 UTC m=+475.958267920" Aug 04 09:48:27.480426 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:27.480297 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:27.566863 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:27.566816 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-6cc2b"] Aug 04 09:48:27.667484 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:27.667448 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:29.206856 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:29.206810 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b54df94b4-6cc2b" podUID="b0286333-8b99-45ca-b848-1191e518c690" containerName="authorino" containerID="cri-o://60161897b5ed4002912d358ca4b2249f20bf5e24b591aa6b5b6d9fb9210c9b17" gracePeriod=30 Aug 04 09:48:29.466696 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:29.466625 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-6cc2b" Aug 04 09:48:29.525457 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:29.525420 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ln88v\" (UniqueName: \"kubernetes.io/projected/b0286333-8b99-45ca-b848-1191e518c690-kube-api-access-ln88v\") pod \"b0286333-8b99-45ca-b848-1191e518c690\" (UID: \"b0286333-8b99-45ca-b848-1191e518c690\") " Aug 04 09:48:29.527485 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:29.527451 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0286333-8b99-45ca-b848-1191e518c690-kube-api-access-ln88v" (OuterVolumeSpecName: "kube-api-access-ln88v") pod "b0286333-8b99-45ca-b848-1191e518c690" (UID: "b0286333-8b99-45ca-b848-1191e518c690"). InnerVolumeSpecName "kube-api-access-ln88v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 09:48:29.626779 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:29.626731 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ln88v\" (UniqueName: \"kubernetes.io/projected/b0286333-8b99-45ca-b848-1191e518c690-kube-api-access-ln88v\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 09:48:30.214123 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:30.214087 2572 generic.go:356] "Generic (PLEG): container finished" podID="b0286333-8b99-45ca-b848-1191e518c690" containerID="60161897b5ed4002912d358ca4b2249f20bf5e24b591aa6b5b6d9fb9210c9b17" exitCode=0 Aug 04 09:48:30.214570 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:30.214131 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-6cc2b" event={"ID":"b0286333-8b99-45ca-b848-1191e518c690","Type":"ContainerDied","Data":"60161897b5ed4002912d358ca4b2249f20bf5e24b591aa6b5b6d9fb9210c9b17"} Aug 04 09:48:30.214570 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:30.214163 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-6cc2b" event={"ID":"b0286333-8b99-45ca-b848-1191e518c690","Type":"ContainerDied","Data":"2b5ad5f47506db046903f9225ac73e7310c77ac4c60db3821538a6f1c6cbd94d"} Aug 04 09:48:30.214570 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:30.214184 2572 scope.go:117] "RemoveContainer" containerID="60161897b5ed4002912d358ca4b2249f20bf5e24b591aa6b5b6d9fb9210c9b17" Aug 04 09:48:30.214570 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:30.214133 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-6cc2b" Aug 04 09:48:30.223050 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:30.223025 2572 scope.go:117] "RemoveContainer" containerID="60161897b5ed4002912d358ca4b2249f20bf5e24b591aa6b5b6d9fb9210c9b17" Aug 04 09:48:30.223356 ip-10-0-129-241 kubenswrapper[2572]: E0804 09:48:30.223335 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60161897b5ed4002912d358ca4b2249f20bf5e24b591aa6b5b6d9fb9210c9b17\": container with ID starting with 60161897b5ed4002912d358ca4b2249f20bf5e24b591aa6b5b6d9fb9210c9b17 not found: ID does not exist" containerID="60161897b5ed4002912d358ca4b2249f20bf5e24b591aa6b5b6d9fb9210c9b17" Aug 04 09:48:30.223416 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:30.223365 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60161897b5ed4002912d358ca4b2249f20bf5e24b591aa6b5b6d9fb9210c9b17"} err="failed to get container status \"60161897b5ed4002912d358ca4b2249f20bf5e24b591aa6b5b6d9fb9210c9b17\": rpc error: code = NotFound desc = could not find container \"60161897b5ed4002912d358ca4b2249f20bf5e24b591aa6b5b6d9fb9210c9b17\": container with ID starting with 60161897b5ed4002912d358ca4b2249f20bf5e24b591aa6b5b6d9fb9210c9b17 not found: ID does not exist" Aug 04 09:48:30.233761 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:30.233732 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-6cc2b"] Aug 04 09:48:30.236037 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:30.236015 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b54df94b4-6cc2b"] Aug 04 09:48:31.779120 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:31.779085 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0286333-8b99-45ca-b848-1191e518c690" path="/var/lib/kubelet/pods/b0286333-8b99-45ca-b848-1191e518c690/volumes" Aug 04 09:48:34.008304 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:34.008268 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:34.125781 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:34.125747 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:39.572374 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:39.572338 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:39.689488 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:39.689457 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:40.154673 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:40.154629 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:51.960987 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:51.960952 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:52.513141 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:52.513086 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:48:59.732368 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:48:59.732334 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:49:00.064333 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:00.064244 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:49:00.168196 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:00.168159 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:49:00.377411 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:00.377318 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:49:03.563441 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:03.563406 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:49:03.679421 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:03.679384 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:49:35.451528 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:35.451442 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:49:35.601153 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:35.601111 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:49:37.076580 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:37.076544 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:49:37.231005 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:37.230971 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:49:37.968753 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:37.968719 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:49:38.110099 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:38.110068 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:49:38.282548 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:38.282469 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-gj8d6"] Aug 04 09:49:38.282878 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:38.282865 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0286333-8b99-45ca-b848-1191e518c690" containerName="authorino" Aug 04 09:49:38.282927 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:38.282880 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0286333-8b99-45ca-b848-1191e518c690" containerName="authorino" Aug 04 09:49:38.282963 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:38.282956 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0286333-8b99-45ca-b848-1191e518c690" containerName="authorino" Aug 04 09:49:38.286009 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:38.285991 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-gj8d6" Aug 04 09:49:38.291705 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:38.291684 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-sgk5v\"" Aug 04 09:49:38.291875 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:38.291685 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Aug 04 09:49:38.302670 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:38.302639 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-gj8d6"] Aug 04 09:49:38.365261 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:38.365215 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rpqj\" (UniqueName: \"kubernetes.io/projected/92d63f31-9830-471b-9e5b-2707def24198-kube-api-access-7rpqj\") pod \"payload-processing-5cc6cd444b-gj8d6\" (UID: \"92d63f31-9830-471b-9e5b-2707def24198\") " pod="openshift-ingress/payload-processing-5cc6cd444b-gj8d6" Aug 04 09:49:38.365460 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:38.365330 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92d63f31-9830-471b-9e5b-2707def24198-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-gj8d6\" (UID: \"92d63f31-9830-471b-9e5b-2707def24198\") " pod="openshift-ingress/payload-processing-5cc6cd444b-gj8d6" Aug 04 09:49:38.466677 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:38.466632 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rpqj\" (UniqueName: \"kubernetes.io/projected/92d63f31-9830-471b-9e5b-2707def24198-kube-api-access-7rpqj\") pod \"payload-processing-5cc6cd444b-gj8d6\" (UID: \"92d63f31-9830-471b-9e5b-2707def24198\") " pod="openshift-ingress/payload-processing-5cc6cd444b-gj8d6" Aug 04 09:49:38.467389 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:38.467363 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92d63f31-9830-471b-9e5b-2707def24198-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-gj8d6\" (UID: \"92d63f31-9830-471b-9e5b-2707def24198\") " pod="openshift-ingress/payload-processing-5cc6cd444b-gj8d6" Aug 04 09:49:38.468687 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:38.468656 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92d63f31-9830-471b-9e5b-2707def24198-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-gj8d6\" (UID: \"92d63f31-9830-471b-9e5b-2707def24198\") " pod="openshift-ingress/payload-processing-5cc6cd444b-gj8d6" Aug 04 09:49:38.481032 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:38.480956 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rpqj\" (UniqueName: \"kubernetes.io/projected/92d63f31-9830-471b-9e5b-2707def24198-kube-api-access-7rpqj\") pod \"payload-processing-5cc6cd444b-gj8d6\" (UID: \"92d63f31-9830-471b-9e5b-2707def24198\") " pod="openshift-ingress/payload-processing-5cc6cd444b-gj8d6" Aug 04 09:49:38.597276 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:38.597181 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-gj8d6" Aug 04 09:49:38.790395 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:38.790348 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-gj8d6"] Aug 04 09:49:38.795789 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:49:38.795748 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92d63f31_9830_471b_9e5b_2707def24198.slice/crio-45068aebfb4635a96a66884c8cd7cc03c9f668db7b7f673cf6da3d04edd14d79 WatchSource:0}: Error finding container 45068aebfb4635a96a66884c8cd7cc03c9f668db7b7f673cf6da3d04edd14d79: Status 404 returned error can't find the container with id 45068aebfb4635a96a66884c8cd7cc03c9f668db7b7f673cf6da3d04edd14d79 Aug 04 09:49:39.598375 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:39.598340 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-gj8d6" event={"ID":"92d63f31-9830-471b-9e5b-2707def24198","Type":"ContainerStarted","Data":"45068aebfb4635a96a66884c8cd7cc03c9f668db7b7f673cf6da3d04edd14d79"} Aug 04 09:49:41.612636 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:41.612596 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-gj8d6" event={"ID":"92d63f31-9830-471b-9e5b-2707def24198","Type":"ContainerStarted","Data":"8b7cf46885b5fe1515140458d6629f6e9c7065886187c0f49be09baaa6911e3e"} Aug 04 09:49:41.613168 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:41.612740 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-gj8d6" Aug 04 09:49:41.636178 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:41.636125 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-gj8d6" podStartSLOduration=1.010184018 podStartE2EDuration="3.636110061s" totalImagesPullingTime="2.625926043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:49:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:49:41.633068013 +0000 UTC m=+550.377202444" watchObservedRunningTime="2026-08-04 09:49:41.636110061 +0000 UTC m=+550.380244480" Aug 04 09:49:47.646875 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:47.646840 2572 generic.go:356] "Generic (PLEG): container finished" podID="1c84a000-d9ad-433f-881b-5cb6e5b32088" containerID="ca194bc984854baab518da779d9f1bc54c214ae89ab19d3005734a0ce287a4a8" exitCode=0 Aug 04 09:49:47.647302 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:47.646916 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" event={"ID":"1c84a000-d9ad-433f-881b-5cb6e5b32088","Type":"ContainerDied","Data":"ca194bc984854baab518da779d9f1bc54c214ae89ab19d3005734a0ce287a4a8"} Aug 04 09:49:47.647351 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:47.647321 2572 scope.go:117] "RemoveContainer" containerID="ca194bc984854baab518da779d9f1bc54c214ae89ab19d3005734a0ce287a4a8" Aug 04 09:49:47.648398 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:47.648371 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 09:49:47.648490 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:47.648473 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 09:49:47.648569 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:47.648526 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 09:49:48.053603 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:48.053566 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:49:48.151650 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:48.151616 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:49:48.654042 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:48.654005 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" event={"ID":"1c84a000-d9ad-433f-881b-5cb6e5b32088","Type":"ContainerStarted","Data":"d7fb94a683901c7012abad8eaad9a0196c4b5e2e12396f722553e393cc2c3d5c"} Aug 04 09:49:49.032049 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:49.032013 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:49:49.032976 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:49.032945 2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-6284q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Aug 04 09:49:49.033111 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:49.033025 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" podUID="1c84a000-d9ad-433f-881b-5cb6e5b32088" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Aug 04 09:49:50.032077 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:50.032038 2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-6284q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Aug 04 09:49:50.032479 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:50.032096 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" podUID="1c84a000-d9ad-433f-881b-5cb6e5b32088" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Aug 04 09:49:51.032051 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:51.032013 2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-6284q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Aug 04 09:49:51.032239 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:51.032071 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" podUID="1c84a000-d9ad-433f-881b-5cb6e5b32088" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Aug 04 09:49:51.794859 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:51.794812 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:49:52.031872 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:52.031838 2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-6284q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Aug 04 09:49:52.032035 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:52.031904 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" podUID="1c84a000-d9ad-433f-881b-5cb6e5b32088" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Aug 04 09:49:52.620208 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:52.620173 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-gj8d6" Aug 04 09:49:53.032128 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:53.032091 2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-6284q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Aug 04 09:49:53.032317 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:53.032152 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" podUID="1c84a000-d9ad-433f-881b-5cb6e5b32088" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Aug 04 09:49:53.180114 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:53.180062 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:49:54.035602 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:54.035565 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:49:54.036700 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:49:54.036680 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6284q" Aug 04 09:50:00.088930 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:00.088897 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw"] Aug 04 09:50:00.092605 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:00.092587 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw" Aug 04 09:50:00.100820 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:00.100776 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Aug 04 09:50:00.100820 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:00.100813 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Aug 04 09:50:00.100996 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:00.100775 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Aug 04 09:50:00.100996 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:00.100778 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-z7ff4\"" Aug 04 09:50:00.122153 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:00.122117 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw"] Aug 04 09:50:00.184427 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:00.184392 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svznw\" (UniqueName: \"kubernetes.io/projected/bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2-kube-api-access-svznw\") pod \"e2e-distinct-simulated-kserve-59975d69f9-79hxw\" (UID: \"bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw" Aug 04 09:50:00.184612 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:00.184503 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-79hxw\" (UID: \"bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw" Aug 04 09:50:00.184612 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:00.184566 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-79hxw\" (UID: \"bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw" Aug 04 09:50:00.184612 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:00.184601 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-79hxw\" (UID: \"bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw" Aug 04 09:50:00.184858 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:00.184626 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-79hxw\" (UID: \"bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw" Aug 04 09:50:00.184858 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:00.184723 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-79hxw\" (UID: \"bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw" Aug 04 09:50:00.263322 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:00.263284 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:50:00.285914 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:00.285879 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-79hxw\" (UID: \"bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw" Aug 04 09:50:00.286130 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:00.285939 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svznw\" (UniqueName: \"kubernetes.io/projected/bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2-kube-api-access-svznw\") pod \"e2e-distinct-simulated-kserve-59975d69f9-79hxw\" (UID: \"bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw" Aug 04 09:50:00.286130 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:00.286015 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-79hxw\" (UID: \"bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw" Aug 04 09:50:00.286130 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:00.286033 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-79hxw\" (UID: \"bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw" Aug 04 09:50:00.286130 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:00.286055 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-79hxw\" (UID: \"bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw" Aug 04 09:50:00.286130 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:00.286079 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-79hxw\" (UID: \"bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw" Aug 04 09:50:00.286470 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:00.286445 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-79hxw\" (UID: \"bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw" Aug 04 09:50:00.286537 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:00.286476 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-79hxw\" (UID: \"bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw" Aug 04 09:50:00.286537 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:00.286528 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-79hxw\" (UID: \"bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw" Aug 04 09:50:00.288345 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:00.288319 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-79hxw\" (UID: \"bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw" Aug 04 09:50:00.289107 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:00.289085 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-79hxw\" (UID: \"bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw" Aug 04 09:50:00.298186 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:00.298162 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svznw\" (UniqueName: \"kubernetes.io/projected/bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2-kube-api-access-svznw\") pod \"e2e-distinct-simulated-kserve-59975d69f9-79hxw\" (UID: \"bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw" Aug 04 09:50:00.403087 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:00.403002 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw" Aug 04 09:50:00.562986 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:00.562952 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw"] Aug 04 09:50:00.564556 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:50:00.564530 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb85e7e2_bb03_4ea3_b44b_1cf5091c1ac2.slice/crio-2937bee91ba1c75a9015b69df202f8f7db92ebb7b7ecb286326e5542d9a03419 WatchSource:0}: Error finding container 2937bee91ba1c75a9015b69df202f8f7db92ebb7b7ecb286326e5542d9a03419: Status 404 returned error can't find the container with id 2937bee91ba1c75a9015b69df202f8f7db92ebb7b7ecb286326e5542d9a03419 Aug 04 09:50:00.739182 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:00.739146 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw" event={"ID":"bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2","Type":"ContainerStarted","Data":"2937bee91ba1c75a9015b69df202f8f7db92ebb7b7ecb286326e5542d9a03419"} Aug 04 09:50:02.753601 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:02.753560 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw" event={"ID":"bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2","Type":"ContainerStarted","Data":"2f2ba32de5b0c6d9499f57d9f2d767026f637b73de2be19628800a7fc1abf345"} Aug 04 09:50:07.343046 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:07.343013 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:50:10.403349 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:10.403308 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw" Aug 04 09:50:10.403761 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:10.403374 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw" Aug 04 09:50:10.415876 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:10.415851 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw" Aug 04 09:50:10.446766 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:10.446713 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw" podStartSLOduration=9.294653702 podStartE2EDuration="10.446696975s" totalImagesPullingTime="1.152043273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:50:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:50:02.780705122 +0000 UTC m=+571.524839550" watchObservedRunningTime="2026-08-04 09:50:10.446696975 +0000 UTC m=+579.190831428" Aug 04 09:50:10.811634 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:10.811605 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-79hxw" Aug 04 09:50:15.486557 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:15.486518 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:50:20.837137 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:20.837103 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:50:31.669342 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:31.669309 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sdsr_7858cfca-73db-4ee1-9882-c13444f112a5/ovn-acl-logging/0.log" Aug 04 09:50:31.669815 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:31.669782 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sdsr_7858cfca-73db-4ee1-9882-c13444f112a5/ovn-acl-logging/0.log" Aug 04 09:50:44.036710 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:44.036665 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm"] Aug 04 09:50:44.041199 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:44.041176 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm" Aug 04 09:50:44.045992 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:44.045961 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Aug 04 09:50:44.060228 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:44.060202 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm"] Aug 04 09:50:44.105540 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:44.105507 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5121391e-8a7a-4897-8bf2-1a8c82d5632b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gk4wm\" (UID: \"5121391e-8a7a-4897-8bf2-1a8c82d5632b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm" Aug 04 09:50:44.105715 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:44.105567 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5121391e-8a7a-4897-8bf2-1a8c82d5632b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gk4wm\" (UID: \"5121391e-8a7a-4897-8bf2-1a8c82d5632b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm" Aug 04 09:50:44.105715 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:44.105622 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5121391e-8a7a-4897-8bf2-1a8c82d5632b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gk4wm\" (UID: \"5121391e-8a7a-4897-8bf2-1a8c82d5632b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm" Aug 04 09:50:44.105715 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:44.105655 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5121391e-8a7a-4897-8bf2-1a8c82d5632b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gk4wm\" (UID: \"5121391e-8a7a-4897-8bf2-1a8c82d5632b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm" Aug 04 09:50:44.105715 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:44.105673 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwpmf\" (UniqueName: \"kubernetes.io/projected/5121391e-8a7a-4897-8bf2-1a8c82d5632b-kube-api-access-rwpmf\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gk4wm\" (UID: \"5121391e-8a7a-4897-8bf2-1a8c82d5632b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm" Aug 04 09:50:44.105908 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:44.105751 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5121391e-8a7a-4897-8bf2-1a8c82d5632b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gk4wm\" (UID: \"5121391e-8a7a-4897-8bf2-1a8c82d5632b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm" Aug 04 09:50:44.108392 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:44.108359 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:50:44.207347 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:44.207304 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5121391e-8a7a-4897-8bf2-1a8c82d5632b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gk4wm\" (UID: \"5121391e-8a7a-4897-8bf2-1a8c82d5632b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm" Aug 04 09:50:44.207559 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:44.207359 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwpmf\" (UniqueName: \"kubernetes.io/projected/5121391e-8a7a-4897-8bf2-1a8c82d5632b-kube-api-access-rwpmf\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gk4wm\" (UID: \"5121391e-8a7a-4897-8bf2-1a8c82d5632b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm" Aug 04 09:50:44.207559 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:44.207433 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5121391e-8a7a-4897-8bf2-1a8c82d5632b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gk4wm\" (UID: \"5121391e-8a7a-4897-8bf2-1a8c82d5632b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm" Aug 04 09:50:44.207559 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:44.207487 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5121391e-8a7a-4897-8bf2-1a8c82d5632b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gk4wm\" (UID: \"5121391e-8a7a-4897-8bf2-1a8c82d5632b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm" Aug 04 09:50:44.207559 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:44.207543 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5121391e-8a7a-4897-8bf2-1a8c82d5632b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gk4wm\" (UID: \"5121391e-8a7a-4897-8bf2-1a8c82d5632b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm" Aug 04 09:50:44.207835 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:44.207589 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5121391e-8a7a-4897-8bf2-1a8c82d5632b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gk4wm\" (UID: \"5121391e-8a7a-4897-8bf2-1a8c82d5632b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm" Aug 04 09:50:44.207985 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:44.207960 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5121391e-8a7a-4897-8bf2-1a8c82d5632b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gk4wm\" (UID: \"5121391e-8a7a-4897-8bf2-1a8c82d5632b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm" Aug 04 09:50:44.208283 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:44.208256 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5121391e-8a7a-4897-8bf2-1a8c82d5632b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gk4wm\" (UID: \"5121391e-8a7a-4897-8bf2-1a8c82d5632b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm" Aug 04 09:50:44.208507 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:44.208336 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5121391e-8a7a-4897-8bf2-1a8c82d5632b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gk4wm\" (UID: \"5121391e-8a7a-4897-8bf2-1a8c82d5632b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm" Aug 04 09:50:44.210073 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:44.210041 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5121391e-8a7a-4897-8bf2-1a8c82d5632b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gk4wm\" (UID: \"5121391e-8a7a-4897-8bf2-1a8c82d5632b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm" Aug 04 09:50:44.210351 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:44.210330 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5121391e-8a7a-4897-8bf2-1a8c82d5632b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gk4wm\" (UID: \"5121391e-8a7a-4897-8bf2-1a8c82d5632b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm" Aug 04 09:50:44.217490 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:44.217461 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwpmf\" (UniqueName: \"kubernetes.io/projected/5121391e-8a7a-4897-8bf2-1a8c82d5632b-kube-api-access-rwpmf\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gk4wm\" (UID: \"5121391e-8a7a-4897-8bf2-1a8c82d5632b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm" Aug 04 09:50:44.352060 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:44.351960 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm" Aug 04 09:50:44.496622 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:44.496590 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm"] Aug 04 09:50:44.497645 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:50:44.497610 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5121391e_8a7a_4897_8bf2_1a8c82d5632b.slice/crio-21eeb71672d7a476e026b0814578865f2fea1af3fca5fde5fb03aa3c3113cf88 WatchSource:0}: Error finding container 21eeb71672d7a476e026b0814578865f2fea1af3fca5fde5fb03aa3c3113cf88: Status 404 returned error can't find the container with id 21eeb71672d7a476e026b0814578865f2fea1af3fca5fde5fb03aa3c3113cf88 Aug 04 09:50:45.002506 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:45.002416 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm" event={"ID":"5121391e-8a7a-4897-8bf2-1a8c82d5632b","Type":"ContainerStarted","Data":"8870c13217ce6920a64ffdb9d93085e825c2f7154b314aafd3dbd1c5f1b7e266"} Aug 04 09:50:45.002506 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:45.002470 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm" event={"ID":"5121391e-8a7a-4897-8bf2-1a8c82d5632b","Type":"ContainerStarted","Data":"21eeb71672d7a476e026b0814578865f2fea1af3fca5fde5fb03aa3c3113cf88"} Aug 04 09:50:45.025616 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:45.025558 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm" podStartSLOduration=0.813624038 podStartE2EDuration="1.025541632s" totalImagesPullingTime="211.917594ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:50:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:50:45.023635882 +0000 UTC m=+613.767770338" watchObservedRunningTime="2026-08-04 09:50:45.025541632 +0000 UTC m=+613.769676050" Aug 04 09:50:45.238652 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:45.238617 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:50:45.339585 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:45.339507 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:50:48.361632 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:48.361600 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv"] Aug 04 09:50:48.365193 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:48.365174 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv" Aug 04 09:50:48.369183 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:48.369160 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Aug 04 09:50:48.399669 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:48.399635 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv"] Aug 04 09:50:48.444358 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:48.444319 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:50:48.447509 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:48.447481 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjzv9\" (UniqueName: \"kubernetes.io/projected/28d347db-4831-422b-9ded-03103b007983-kube-api-access-pjzv9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv\" (UID: \"28d347db-4831-422b-9ded-03103b007983\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv" Aug 04 09:50:48.447606 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:48.447520 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28d347db-4831-422b-9ded-03103b007983-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv\" (UID: \"28d347db-4831-422b-9ded-03103b007983\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv" Aug 04 09:50:48.447606 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:48.447549 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28d347db-4831-422b-9ded-03103b007983-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv\" (UID: \"28d347db-4831-422b-9ded-03103b007983\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv" Aug 04 09:50:48.447682 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:48.447648 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28d347db-4831-422b-9ded-03103b007983-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv\" (UID: \"28d347db-4831-422b-9ded-03103b007983\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv" Aug 04 09:50:48.447742 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:48.447729 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28d347db-4831-422b-9ded-03103b007983-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv\" (UID: \"28d347db-4831-422b-9ded-03103b007983\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv" Aug 04 09:50:48.447777 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:48.447759 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28d347db-4831-422b-9ded-03103b007983-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv\" (UID: \"28d347db-4831-422b-9ded-03103b007983\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv" Aug 04 09:50:48.549243 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:48.549204 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28d347db-4831-422b-9ded-03103b007983-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv\" (UID: \"28d347db-4831-422b-9ded-03103b007983\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv" Aug 04 09:50:48.549243 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:48.549248 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28d347db-4831-422b-9ded-03103b007983-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv\" (UID: \"28d347db-4831-422b-9ded-03103b007983\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv" Aug 04 09:50:48.549480 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:48.549298 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjzv9\" (UniqueName: \"kubernetes.io/projected/28d347db-4831-422b-9ded-03103b007983-kube-api-access-pjzv9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv\" (UID: \"28d347db-4831-422b-9ded-03103b007983\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv" Aug 04 09:50:48.549480 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:48.549325 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28d347db-4831-422b-9ded-03103b007983-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv\" (UID: \"28d347db-4831-422b-9ded-03103b007983\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv" Aug 04 09:50:48.549480 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:48.549343 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28d347db-4831-422b-9ded-03103b007983-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv\" (UID: \"28d347db-4831-422b-9ded-03103b007983\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv" Aug 04 09:50:48.549480 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:48.549376 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28d347db-4831-422b-9ded-03103b007983-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv\" (UID: \"28d347db-4831-422b-9ded-03103b007983\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv" Aug 04 09:50:48.549764 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:48.549740 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28d347db-4831-422b-9ded-03103b007983-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv\" (UID: \"28d347db-4831-422b-9ded-03103b007983\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv" Aug 04 09:50:48.549882 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:48.549776 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28d347db-4831-422b-9ded-03103b007983-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv\" (UID: \"28d347db-4831-422b-9ded-03103b007983\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv" Aug 04 09:50:48.549882 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:48.549870 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28d347db-4831-422b-9ded-03103b007983-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv\" (UID: \"28d347db-4831-422b-9ded-03103b007983\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv" Aug 04 09:50:48.551600 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:48.551582 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28d347db-4831-422b-9ded-03103b007983-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv\" (UID: \"28d347db-4831-422b-9ded-03103b007983\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv" Aug 04 09:50:48.551704 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:48.551688 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28d347db-4831-422b-9ded-03103b007983-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv\" (UID: \"28d347db-4831-422b-9ded-03103b007983\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv" Aug 04 09:50:48.559883 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:48.559847 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjzv9\" (UniqueName: \"kubernetes.io/projected/28d347db-4831-422b-9ded-03103b007983-kube-api-access-pjzv9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv\" (UID: \"28d347db-4831-422b-9ded-03103b007983\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv" Aug 04 09:50:48.675939 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:48.675840 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv" Aug 04 09:50:48.839157 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:48.839135 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv"] Aug 04 09:50:48.840787 ip-10-0-129-241 kubenswrapper[2572]: W0804 09:50:48.840755 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod28d347db_4831_422b_9ded_03103b007983.slice/crio-e58ab966841f683fc546e4dec9db52c49dd6e773fd2c7e683ef6c4167c3ccaa7 WatchSource:0}: Error finding container e58ab966841f683fc546e4dec9db52c49dd6e773fd2c7e683ef6c4167c3ccaa7: Status 404 returned error can't find the container with id e58ab966841f683fc546e4dec9db52c49dd6e773fd2c7e683ef6c4167c3ccaa7 Aug 04 09:50:49.027201 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:49.027165 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv" event={"ID":"28d347db-4831-422b-9ded-03103b007983","Type":"ContainerStarted","Data":"e58ab966841f683fc546e4dec9db52c49dd6e773fd2c7e683ef6c4167c3ccaa7"} Aug 04 09:50:49.714720 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:49.714684 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:50:50.033960 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:50.033862 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv" event={"ID":"28d347db-4831-422b-9ded-03103b007983","Type":"ContainerStarted","Data":"9bcd4a4ed602773fac6fa8b2431294c05a456b198165a79402c30e4046b29507"} Aug 04 09:50:50.055023 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:50.054971 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv" podStartSLOduration=1.851196957 podStartE2EDuration="2.054955045s" totalImagesPullingTime="203.758088ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 09:50:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 09:50:50.052716259 +0000 UTC m=+618.796850690" watchObservedRunningTime="2026-08-04 09:50:50.054955045 +0000 UTC m=+618.799089463" Aug 04 09:50:54.352557 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:54.352459 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm" Aug 04 09:50:54.352557 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:54.352552 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm" Aug 04 09:50:54.365130 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:54.365085 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm" Aug 04 09:50:55.075411 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:55.075372 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gk4wm" Aug 04 09:50:58.676131 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:58.676092 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv" Aug 04 09:50:58.676516 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:58.676151 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv" Aug 04 09:50:58.688568 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:58.688542 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv" Aug 04 09:50:59.098707 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:50:59.098676 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv" Aug 04 09:51:20.146810 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:51:20.146768 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:51:43.463126 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:51:43.463092 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:51:49.284786 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:51:49.284748 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:51:49.486217 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:51:49.486181 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:51:50.640677 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:51:50.640646 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:51:50.798986 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:51:50.798954 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:52:42.647605 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:52:42.647558 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:53:58.677686 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:53:58.677583 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:53:59.938857 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:53:59.938817 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:54:38.757500 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:54:38.757461 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:54:38.857525 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:54:38.857488 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:55:02.744323 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:55:02.744288 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:55:02.970141 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:55:02.970101 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:55:03.125158 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:55:03.125070 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:55:09.286688 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:55:09.286649 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:55:09.636874 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:55:09.636754 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:55:09.788745 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:55:09.788709 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:55:09.958097 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:55:09.958058 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:55:16.113843 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:55:16.113804 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:55:16.212868 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:55:16.212829 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:55:25.033726 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:55:25.033644 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:55:25.116025 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:55:25.115994 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 09:55:31.707738 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:55:31.707711 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sdsr_7858cfca-73db-4ee1-9882-c13444f112a5/ovn-acl-logging/0.log" Aug 04 09:55:31.708426 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:55:31.708402 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sdsr_7858cfca-73db-4ee1-9882-c13444f112a5/ovn-acl-logging/0.log" Aug 04 09:58:01.438955 ip-10-0-129-241 kubenswrapper[2572]: I0804 09:58:01.438905 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:00:23.161760 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:00:23.161717 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:00:23.252339 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:00:23.252292 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:00:31.747517 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:00:31.747488 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sdsr_7858cfca-73db-4ee1-9882-c13444f112a5/ovn-acl-logging/0.log" Aug 04 10:00:31.751870 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:00:31.751847 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sdsr_7858cfca-73db-4ee1-9882-c13444f112a5/ovn-acl-logging/0.log" Aug 04 10:00:33.644555 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:00:33.644516 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lp98f"] Aug 04 10:00:33.645040 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:00:33.644737 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lp98f" podUID="210a1804-8468-4310-9779-e7c9a3d3eeb7" containerName="manager" containerID="cri-o://82339c8473d646f6f37fdbcafaaa1e682aafb6e1aae989d22c8c4a6e062406c1" gracePeriod=10 Aug 04 10:00:33.912368 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:00:33.912340 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lp98f" Aug 04 10:00:34.053758 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:00:34.053726 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfgnd\" (UniqueName: \"kubernetes.io/projected/210a1804-8468-4310-9779-e7c9a3d3eeb7-kube-api-access-wfgnd\") pod \"210a1804-8468-4310-9779-e7c9a3d3eeb7\" (UID: \"210a1804-8468-4310-9779-e7c9a3d3eeb7\") " Aug 04 10:00:34.054062 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:00:34.053771 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/210a1804-8468-4310-9779-e7c9a3d3eeb7-extensions-socket-volume\") pod \"210a1804-8468-4310-9779-e7c9a3d3eeb7\" (UID: \"210a1804-8468-4310-9779-e7c9a3d3eeb7\") " Aug 04 10:00:34.054182 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:00:34.054160 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/210a1804-8468-4310-9779-e7c9a3d3eeb7-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "210a1804-8468-4310-9779-e7c9a3d3eeb7" (UID: "210a1804-8468-4310-9779-e7c9a3d3eeb7"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:00:34.055789 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:00:34.055764 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/210a1804-8468-4310-9779-e7c9a3d3eeb7-kube-api-access-wfgnd" (OuterVolumeSpecName: "kube-api-access-wfgnd") pod "210a1804-8468-4310-9779-e7c9a3d3eeb7" (UID: "210a1804-8468-4310-9779-e7c9a3d3eeb7"). InnerVolumeSpecName "kube-api-access-wfgnd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:00:34.155316 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:00:34.155211 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfgnd\" (UniqueName: \"kubernetes.io/projected/210a1804-8468-4310-9779-e7c9a3d3eeb7-kube-api-access-wfgnd\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 10:00:34.155316 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:00:34.155259 2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/210a1804-8468-4310-9779-e7c9a3d3eeb7-extensions-socket-volume\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 10:00:34.373386 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:00:34.373351 2572 generic.go:356] "Generic (PLEG): container finished" podID="210a1804-8468-4310-9779-e7c9a3d3eeb7" containerID="82339c8473d646f6f37fdbcafaaa1e682aafb6e1aae989d22c8c4a6e062406c1" exitCode=0 Aug 04 10:00:34.373560 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:00:34.373408 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lp98f" Aug 04 10:00:34.373560 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:00:34.373431 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lp98f" event={"ID":"210a1804-8468-4310-9779-e7c9a3d3eeb7","Type":"ContainerDied","Data":"82339c8473d646f6f37fdbcafaaa1e682aafb6e1aae989d22c8c4a6e062406c1"} Aug 04 10:00:34.373560 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:00:34.373508 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lp98f" event={"ID":"210a1804-8468-4310-9779-e7c9a3d3eeb7","Type":"ContainerDied","Data":"024edd440f6c9c078f217c80190b1e103ccea24d24f6dfa26ed04670e931bbcf"} Aug 04 10:00:34.373560 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:00:34.373531 2572 scope.go:117] "RemoveContainer" containerID="82339c8473d646f6f37fdbcafaaa1e682aafb6e1aae989d22c8c4a6e062406c1" Aug 04 10:00:34.384329 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:00:34.384312 2572 scope.go:117] "RemoveContainer" containerID="82339c8473d646f6f37fdbcafaaa1e682aafb6e1aae989d22c8c4a6e062406c1" Aug 04 10:00:34.384634 ip-10-0-129-241 kubenswrapper[2572]: E0804 10:00:34.384609 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82339c8473d646f6f37fdbcafaaa1e682aafb6e1aae989d22c8c4a6e062406c1\": container with ID starting with 82339c8473d646f6f37fdbcafaaa1e682aafb6e1aae989d22c8c4a6e062406c1 not found: ID does not exist" containerID="82339c8473d646f6f37fdbcafaaa1e682aafb6e1aae989d22c8c4a6e062406c1" Aug 04 10:00:34.384694 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:00:34.384644 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82339c8473d646f6f37fdbcafaaa1e682aafb6e1aae989d22c8c4a6e062406c1"} err="failed to get container status \"82339c8473d646f6f37fdbcafaaa1e682aafb6e1aae989d22c8c4a6e062406c1\": rpc error: code = NotFound desc = could not find container \"82339c8473d646f6f37fdbcafaaa1e682aafb6e1aae989d22c8c4a6e062406c1\": container with ID starting with 82339c8473d646f6f37fdbcafaaa1e682aafb6e1aae989d22c8c4a6e062406c1 not found: ID does not exist" Aug 04 10:00:34.399720 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:00:34.399685 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lp98f"] Aug 04 10:00:34.406481 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:00:34.406406 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-lp98f"] Aug 04 10:00:35.779206 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:00:35.779173 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="210a1804-8468-4310-9779-e7c9a3d3eeb7" path="/var/lib/kubelet/pods/210a1804-8468-4310-9779-e7c9a3d3eeb7/volumes" Aug 04 10:01:13.758165 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:01:13.758125 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:01:13.840317 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:01:13.840282 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:01:34.395811 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:01:34.395773 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8sk9p"] Aug 04 10:01:34.396196 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:01:34.396184 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="210a1804-8468-4310-9779-e7c9a3d3eeb7" containerName="manager" Aug 04 10:01:34.396196 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:01:34.396195 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="210a1804-8468-4310-9779-e7c9a3d3eeb7" containerName="manager" Aug 04 10:01:34.396269 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:01:34.396263 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="210a1804-8468-4310-9779-e7c9a3d3eeb7" containerName="manager" Aug 04 10:01:34.399843 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:01:34.399825 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8sk9p" Aug 04 10:01:34.403904 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:01:34.403882 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-wp5hm\"" Aug 04 10:01:34.412670 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:01:34.412642 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8sk9p"] Aug 04 10:01:34.440823 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:01:34.440764 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6rwp\" (UniqueName: \"kubernetes.io/projected/1b562b77-39d4-4f31-a447-b6a5d23e8a31-kube-api-access-s6rwp\") pod \"kuadrant-operator-controller-manager-68cf589589-8sk9p\" (UID: \"1b562b77-39d4-4f31-a447-b6a5d23e8a31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8sk9p" Aug 04 10:01:34.441008 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:01:34.440892 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b562b77-39d4-4f31-a447-b6a5d23e8a31-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-8sk9p\" (UID: \"1b562b77-39d4-4f31-a447-b6a5d23e8a31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8sk9p" Aug 04 10:01:34.541686 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:01:34.541648 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b562b77-39d4-4f31-a447-b6a5d23e8a31-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-8sk9p\" (UID: \"1b562b77-39d4-4f31-a447-b6a5d23e8a31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8sk9p" Aug 04 10:01:34.541905 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:01:34.541765 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6rwp\" (UniqueName: \"kubernetes.io/projected/1b562b77-39d4-4f31-a447-b6a5d23e8a31-kube-api-access-s6rwp\") pod \"kuadrant-operator-controller-manager-68cf589589-8sk9p\" (UID: \"1b562b77-39d4-4f31-a447-b6a5d23e8a31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8sk9p" Aug 04 10:01:34.542097 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:01:34.542076 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b562b77-39d4-4f31-a447-b6a5d23e8a31-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-8sk9p\" (UID: \"1b562b77-39d4-4f31-a447-b6a5d23e8a31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8sk9p" Aug 04 10:01:34.552895 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:01:34.552863 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6rwp\" (UniqueName: \"kubernetes.io/projected/1b562b77-39d4-4f31-a447-b6a5d23e8a31-kube-api-access-s6rwp\") pod \"kuadrant-operator-controller-manager-68cf589589-8sk9p\" (UID: \"1b562b77-39d4-4f31-a447-b6a5d23e8a31\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8sk9p" Aug 04 10:01:34.710230 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:01:34.710186 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8sk9p" Aug 04 10:01:34.873005 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:01:34.872978 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8sk9p"] Aug 04 10:01:34.875493 ip-10-0-129-241 kubenswrapper[2572]: W0804 10:01:34.875465 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b562b77_39d4_4f31_a447_b6a5d23e8a31.slice/crio-86733806c23489e5bcdfb5bb69bbbccc82f516c20c2be5be74a2679955816e63 WatchSource:0}: Error finding container 86733806c23489e5bcdfb5bb69bbbccc82f516c20c2be5be74a2679955816e63: Status 404 returned error can't find the container with id 86733806c23489e5bcdfb5bb69bbbccc82f516c20c2be5be74a2679955816e63 Aug 04 10:01:35.735399 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:01:35.735359 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8sk9p" event={"ID":"1b562b77-39d4-4f31-a447-b6a5d23e8a31","Type":"ContainerStarted","Data":"a950e72315fc7ab3a97da710486db720621de207164f39f305f44fced9e7e44b"} Aug 04 10:01:35.735846 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:01:35.735412 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8sk9p" event={"ID":"1b562b77-39d4-4f31-a447-b6a5d23e8a31","Type":"ContainerStarted","Data":"86733806c23489e5bcdfb5bb69bbbccc82f516c20c2be5be74a2679955816e63"} Aug 04 10:01:35.735846 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:01:35.735488 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8sk9p" Aug 04 10:01:35.764197 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:01:35.764138 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8sk9p" podStartSLOduration=1.764121276 podStartE2EDuration="1.764121276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:01:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:01:35.761905808 +0000 UTC m=+1264.506040226" watchObservedRunningTime="2026-08-04 10:01:35.764121276 +0000 UTC m=+1264.508255768" Aug 04 10:01:46.743762 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:01:46.743724 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-8sk9p" Aug 04 10:02:14.903819 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:02:14.903752 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:04:53.129168 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:04:53.129128 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:05:07.000196 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:05:07.000162 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:05:31.795985 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:05:31.795954 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sdsr_7858cfca-73db-4ee1-9882-c13444f112a5/ovn-acl-logging/0.log" Aug 04 10:05:31.801186 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:05:31.801166 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sdsr_7858cfca-73db-4ee1-9882-c13444f112a5/ovn-acl-logging/0.log" Aug 04 10:05:55.496536 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:05:55.496415 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:05:56.288376 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:05:56.288326 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:07:32.407434 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:07:32.407351 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:09:28.491393 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:09:28.491300 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:09:55.644639 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:09:55.644597 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:09:59.487531 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:09:59.487464 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:10:31.834145 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:10:31.834017 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sdsr_7858cfca-73db-4ee1-9882-c13444f112a5/ovn-acl-logging/0.log" Aug 04 10:10:31.839264 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:10:31.839001 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sdsr_7858cfca-73db-4ee1-9882-c13444f112a5/ovn-acl-logging/0.log" Aug 04 10:13:47.989732 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:13:47.989695 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:14:19.822236 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:14:19.822199 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:14:19.918649 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:14:19.918620 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:15:31.870271 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:15:31.870154 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sdsr_7858cfca-73db-4ee1-9882-c13444f112a5/ovn-acl-logging/0.log" Aug 04 10:15:31.876317 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:15:31.876294 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sdsr_7858cfca-73db-4ee1-9882-c13444f112a5/ovn-acl-logging/0.log" Aug 04 10:16:32.082777 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:16:32.082686 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:20:04.107470 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:04.107435 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:20:04.288164 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:04.288121 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:20:04.526991 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:04.526960 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:20:11.255135 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:11.255103 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:20:11.356970 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:11.356937 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:20:11.474020 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:11.473988 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:20:11.895766 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:11.895729 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx"] Aug 04 10:20:11.899616 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:11.899593 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:11.915233 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:11.915189 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-9bae9cd1-openshift-default-dockercfg-dgjwq\"" Aug 04 10:20:11.938279 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:11.938241 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx"] Aug 04 10:20:11.946354 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:11.946312 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9ktp\" (UniqueName: \"kubernetes.io/projected/62a12e28-53aa-4563-95d4-a428c8dd9b4f-kube-api-access-c9ktp\") pod \"e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:11.946556 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:11.946377 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istiod-ca-cert\") pod \"e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:11.946556 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:11.946425 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-workload-socket\") pod \"e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:11.946556 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:11.946501 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istio-podinfo\") pod \"e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:11.946556 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:11.946551 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istio-data\") pod \"e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:11.946784 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:11.946608 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-workload-certs\") pod \"e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:11.946784 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:11.946634 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istio-token\") pod \"e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:11.946784 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:11.946721 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-credential-socket\") pod \"e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:11.946784 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:11.946752 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istio-envoy\") pod \"e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:11.971121 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:11.971085 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:20:12.047911 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:12.047861 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9ktp\" (UniqueName: \"kubernetes.io/projected/62a12e28-53aa-4563-95d4-a428c8dd9b4f-kube-api-access-c9ktp\") pod \"e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:12.047911 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:12.047913 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istiod-ca-cert\") pod \"e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:12.048198 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:12.047940 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-workload-socket\") pod \"e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:12.048198 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:12.047960 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istio-podinfo\") pod \"e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:12.048198 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:12.047984 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istio-data\") pod \"e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:12.048198 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:12.048026 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-workload-certs\") pod \"e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:12.048198 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:12.048157 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istio-token\") pod \"e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:12.048466 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:12.048248 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-credential-socket\") pod \"e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:12.048466 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:12.048283 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istio-envoy\") pod \"e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:12.048466 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:12.048414 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istio-data\") pod \"e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:12.048893 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:12.048870 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istiod-ca-cert\") pod \"e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:12.049018 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:12.049001 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-workload-socket\") pod \"e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:12.049101 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:12.049079 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-workload-certs\") pod \"e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:12.049288 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:12.049263 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-credential-socket\") pod \"e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:12.050691 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:12.050671 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istio-podinfo\") pod \"e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:12.050926 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:12.050905 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istio-envoy\") pod \"e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:12.060314 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:12.060270 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istio-token\") pod \"e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:12.060472 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:12.060328 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9ktp\" (UniqueName: \"kubernetes.io/projected/62a12e28-53aa-4563-95d4-a428c8dd9b4f-kube-api-access-c9ktp\") pod \"e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:12.218973 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:12.218932 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:12.382748 ip-10-0-129-241 kubenswrapper[2572]: W0804 10:20:12.382717 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62a12e28_53aa_4563_95d4_a428c8dd9b4f.slice/crio-a96eb2faf23b5144613e5f82b3371ca7e556eeaed7cd4f627d8cf6f808a43ad5 WatchSource:0}: Error finding container a96eb2faf23b5144613e5f82b3371ca7e556eeaed7cd4f627d8cf6f808a43ad5: Status 404 returned error can't find the container with id a96eb2faf23b5144613e5f82b3371ca7e556eeaed7cd4f627d8cf6f808a43ad5 Aug 04 10:20:12.384160 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:12.384130 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx"] Aug 04 10:20:12.384999 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:12.384965 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 10:20:12.385104 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:12.385047 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 10:20:12.385104 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:12.385091 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 10:20:12.514414 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:12.514273 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:20:13.105856 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:13.105814 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" event={"ID":"62a12e28-53aa-4563-95d4-a428c8dd9b4f","Type":"ContainerStarted","Data":"0d35eaebd2222b9a8d7695dcd8ee061677ee1322095a3a30bd60a050170bdd32"} Aug 04 10:20:13.106060 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:13.105880 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" event={"ID":"62a12e28-53aa-4563-95d4-a428c8dd9b4f","Type":"ContainerStarted","Data":"a96eb2faf23b5144613e5f82b3371ca7e556eeaed7cd4f627d8cf6f808a43ad5"} Aug 04 10:20:13.181687 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:13.181633 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" podStartSLOduration=2.18161154 podStartE2EDuration="2.18161154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 10:20:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:20:13.17978867 +0000 UTC m=+2381.923923088" watchObservedRunningTime="2026-08-04 10:20:13.18161154 +0000 UTC m=+2381.925745938" Aug 04 10:20:13.219702 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:13.219662 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:13.224992 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:13.224965 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:14.111598 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:14.111562 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:14.112658 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:14.112638 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:14.612641 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:14.612606 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:20:14.704559 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:14.704526 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:20:15.750976 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:15.750941 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:20:16.266655 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:16.266609 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:20:16.376435 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:16.376398 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:20:16.589141 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:16.589051 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:20:29.903641 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:29.903597 2572 patch_prober.go:28] interesting pod/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": read tcp 10.134.0.2:47690->10.134.0.54:15021: read: connection reset by peer" start-of-body= Aug 04 10:20:29.904041 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:29.903684 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" podUID="62a12e28-53aa-4563-95d4-a428c8dd9b4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": read tcp 10.134.0.2:47690->10.134.0.54:15021: read: connection reset by peer" Aug 04 10:20:30.219325 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:30.219291 2572 generic.go:356] "Generic (PLEG): container finished" podID="62a12e28-53aa-4563-95d4-a428c8dd9b4f" containerID="0d35eaebd2222b9a8d7695dcd8ee061677ee1322095a3a30bd60a050170bdd32" exitCode=0 Aug 04 10:20:30.219510 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:30.219363 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" event={"ID":"62a12e28-53aa-4563-95d4-a428c8dd9b4f","Type":"ContainerDied","Data":"0d35eaebd2222b9a8d7695dcd8ee061677ee1322095a3a30bd60a050170bdd32"} Aug 04 10:20:30.219792 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:30.219779 2572 scope.go:117] "RemoveContainer" containerID="0d35eaebd2222b9a8d7695dcd8ee061677ee1322095a3a30bd60a050170bdd32" Aug 04 10:20:30.220948 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:30.220917 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 10:20:30.221012 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:30.220990 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 10:20:30.221055 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:30.221023 2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 10:20:31.228294 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:31.228259 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" event={"ID":"62a12e28-53aa-4563-95d4-a428c8dd9b4f","Type":"ContainerStarted","Data":"e1bd28e72889f0aeaeb0a26b19faa8eacbd723400b5bff5a8349305be6b1935e"} Aug 04 10:20:31.915905 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:31.915874 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sdsr_7858cfca-73db-4ee1-9882-c13444f112a5/ovn-acl-logging/0.log" Aug 04 10:20:31.922418 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:31.922392 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sdsr_7858cfca-73db-4ee1-9882-c13444f112a5/ovn-acl-logging/0.log" Aug 04 10:20:32.219420 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:32.219387 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:32.219620 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:32.219592 2572 patch_prober.go:28] interesting pod/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Aug 04 10:20:32.219669 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:32.219649 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" podUID="62a12e28-53aa-4563-95d4-a428c8dd9b4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Aug 04 10:20:33.219414 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:33.219371 2572 patch_prober.go:28] interesting pod/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Aug 04 10:20:33.219838 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:33.219438 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" podUID="62a12e28-53aa-4563-95d4-a428c8dd9b4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Aug 04 10:20:34.220180 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:34.220143 2572 patch_prober.go:28] interesting pod/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Aug 04 10:20:34.220610 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:34.220208 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" podUID="62a12e28-53aa-4563-95d4-a428c8dd9b4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Aug 04 10:20:35.219621 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:35.219579 2572 patch_prober.go:28] interesting pod/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Aug 04 10:20:35.219921 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:35.219644 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" podUID="62a12e28-53aa-4563-95d4-a428c8dd9b4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Aug 04 10:20:36.224940 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:36.224907 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:36.225366 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:36.225280 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:20:36.226213 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:20:36.226192 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:22:51.288327 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:22:51.288296 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:23:28.817778 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:28.817734 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:23:28.921641 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:28.921609 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:23:29.022667 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:29.022613 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:23:29.100266 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:29.100179 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:23:29.257865 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:29.257820 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx"] Aug 04 10:23:29.258148 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:29.258097 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" podUID="62a12e28-53aa-4563-95d4-a428c8dd9b4f" containerName="istio-proxy" containerID="cri-o://e1bd28e72889f0aeaeb0a26b19faa8eacbd723400b5bff5a8349305be6b1935e" gracePeriod=30 Aug 04 10:23:29.263719 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:29.263663 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:23:29.515341 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:29.515309 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:23:29.770697 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:29.770592 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:23:29.870430 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:29.870388 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:23:29.993395 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:29.993364 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:23:30.339108 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:30.339071 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:23:30.656415 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:30.656330 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:23:34.516491 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:34.516462 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:23:34.683892 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:34.683780 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istiod-ca-cert\") pod \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " Aug 04 10:23:34.684073 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:34.683895 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9ktp\" (UniqueName: \"kubernetes.io/projected/62a12e28-53aa-4563-95d4-a428c8dd9b4f-kube-api-access-c9ktp\") pod \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " Aug 04 10:23:34.684073 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:34.683930 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istio-envoy\") pod \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " Aug 04 10:23:34.684073 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:34.683985 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istio-token\") pod \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " Aug 04 10:23:34.684073 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:34.684037 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-workload-socket\") pod \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " Aug 04 10:23:34.684282 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:34.684075 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istio-data\") pod \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " Aug 04 10:23:34.684282 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:34.684111 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-workload-certs\") pod \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " Aug 04 10:23:34.684282 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:34.684154 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istio-podinfo\") pod \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " Aug 04 10:23:34.684282 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:34.684184 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-credential-socket\") pod \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\" (UID: \"62a12e28-53aa-4563-95d4-a428c8dd9b4f\") " Aug 04 10:23:34.684282 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:34.684198 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "62a12e28-53aa-4563-95d4-a428c8dd9b4f" (UID: "62a12e28-53aa-4563-95d4-a428c8dd9b4f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 10:23:34.684537 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:34.684324 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "62a12e28-53aa-4563-95d4-a428c8dd9b4f" (UID: "62a12e28-53aa-4563-95d4-a428c8dd9b4f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:23:34.684537 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:34.684496 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "62a12e28-53aa-4563-95d4-a428c8dd9b4f" (UID: "62a12e28-53aa-4563-95d4-a428c8dd9b4f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:23:34.684537 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:34.684515 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istio-data" (OuterVolumeSpecName: "istio-data") pod "62a12e28-53aa-4563-95d4-a428c8dd9b4f" (UID: "62a12e28-53aa-4563-95d4-a428c8dd9b4f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:23:34.684691 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:34.684546 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "62a12e28-53aa-4563-95d4-a428c8dd9b4f" (UID: "62a12e28-53aa-4563-95d4-a428c8dd9b4f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:23:34.684691 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:34.684558 2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-workload-socket\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 10:23:34.684691 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:34.684575 2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istiod-ca-cert\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 10:23:34.686188 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:34.686162 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62a12e28-53aa-4563-95d4-a428c8dd9b4f-kube-api-access-c9ktp" (OuterVolumeSpecName: "kube-api-access-c9ktp") pod "62a12e28-53aa-4563-95d4-a428c8dd9b4f" (UID: "62a12e28-53aa-4563-95d4-a428c8dd9b4f"). InnerVolumeSpecName "kube-api-access-c9ktp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:23:34.686393 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:34.686368 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "62a12e28-53aa-4563-95d4-a428c8dd9b4f" (UID: "62a12e28-53aa-4563-95d4-a428c8dd9b4f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 10:23:34.686558 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:34.686539 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "62a12e28-53aa-4563-95d4-a428c8dd9b4f" (UID: "62a12e28-53aa-4563-95d4-a428c8dd9b4f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 10:23:34.686628 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:34.686575 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istio-token" (OuterVolumeSpecName: "istio-token") pod "62a12e28-53aa-4563-95d4-a428c8dd9b4f" (UID: "62a12e28-53aa-4563-95d4-a428c8dd9b4f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 10:23:34.786036 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:34.785999 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9ktp\" (UniqueName: \"kubernetes.io/projected/62a12e28-53aa-4563-95d4-a428c8dd9b4f-kube-api-access-c9ktp\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 10:23:34.786036 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:34.786030 2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istio-envoy\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 10:23:34.786036 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:34.786040 2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istio-token\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 10:23:34.786036 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:34.786048 2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istio-data\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 10:23:34.786325 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:34.786056 2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-workload-certs\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 10:23:34.786325 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:34.786063 2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62a12e28-53aa-4563-95d4-a428c8dd9b4f-istio-podinfo\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 10:23:34.786325 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:34.786073 2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62a12e28-53aa-4563-95d4-a428c8dd9b4f-credential-socket\") on node \"ip-10-0-129-241.ec2.internal\" DevicePath \"\"" Aug 04 10:23:35.301165 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:35.301133 2572 generic.go:356] "Generic (PLEG): container finished" podID="62a12e28-53aa-4563-95d4-a428c8dd9b4f" containerID="e1bd28e72889f0aeaeb0a26b19faa8eacbd723400b5bff5a8349305be6b1935e" exitCode=0 Aug 04 10:23:35.301165 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:35.301170 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" event={"ID":"62a12e28-53aa-4563-95d4-a428c8dd9b4f","Type":"ContainerDied","Data":"e1bd28e72889f0aeaeb0a26b19faa8eacbd723400b5bff5a8349305be6b1935e"} Aug 04 10:23:35.301442 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:35.301183 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" Aug 04 10:23:35.301442 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:35.301211 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx" event={"ID":"62a12e28-53aa-4563-95d4-a428c8dd9b4f","Type":"ContainerDied","Data":"a96eb2faf23b5144613e5f82b3371ca7e556eeaed7cd4f627d8cf6f808a43ad5"} Aug 04 10:23:35.301442 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:35.301232 2572 scope.go:117] "RemoveContainer" containerID="e1bd28e72889f0aeaeb0a26b19faa8eacbd723400b5bff5a8349305be6b1935e" Aug 04 10:23:35.311227 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:35.311206 2572 scope.go:117] "RemoveContainer" containerID="0d35eaebd2222b9a8d7695dcd8ee061677ee1322095a3a30bd60a050170bdd32" Aug 04 10:23:35.320426 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:35.320405 2572 scope.go:117] "RemoveContainer" containerID="e1bd28e72889f0aeaeb0a26b19faa8eacbd723400b5bff5a8349305be6b1935e" Aug 04 10:23:35.320719 ip-10-0-129-241 kubenswrapper[2572]: E0804 10:23:35.320693 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1bd28e72889f0aeaeb0a26b19faa8eacbd723400b5bff5a8349305be6b1935e\": container with ID starting with e1bd28e72889f0aeaeb0a26b19faa8eacbd723400b5bff5a8349305be6b1935e not found: ID does not exist" containerID="e1bd28e72889f0aeaeb0a26b19faa8eacbd723400b5bff5a8349305be6b1935e" Aug 04 10:23:35.320765 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:35.320729 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1bd28e72889f0aeaeb0a26b19faa8eacbd723400b5bff5a8349305be6b1935e"} err="failed to get container status \"e1bd28e72889f0aeaeb0a26b19faa8eacbd723400b5bff5a8349305be6b1935e\": rpc error: code = NotFound desc = could not find container \"e1bd28e72889f0aeaeb0a26b19faa8eacbd723400b5bff5a8349305be6b1935e\": container with ID starting with e1bd28e72889f0aeaeb0a26b19faa8eacbd723400b5bff5a8349305be6b1935e not found: ID does not exist" Aug 04 10:23:35.320765 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:35.320748 2572 scope.go:117] "RemoveContainer" containerID="0d35eaebd2222b9a8d7695dcd8ee061677ee1322095a3a30bd60a050170bdd32" Aug 04 10:23:35.321013 ip-10-0-129-241 kubenswrapper[2572]: E0804 10:23:35.320994 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d35eaebd2222b9a8d7695dcd8ee061677ee1322095a3a30bd60a050170bdd32\": container with ID starting with 0d35eaebd2222b9a8d7695dcd8ee061677ee1322095a3a30bd60a050170bdd32 not found: ID does not exist" containerID="0d35eaebd2222b9a8d7695dcd8ee061677ee1322095a3a30bd60a050170bdd32" Aug 04 10:23:35.321079 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:35.321018 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d35eaebd2222b9a8d7695dcd8ee061677ee1322095a3a30bd60a050170bdd32"} err="failed to get container status \"0d35eaebd2222b9a8d7695dcd8ee061677ee1322095a3a30bd60a050170bdd32\": rpc error: code = NotFound desc = could not find container \"0d35eaebd2222b9a8d7695dcd8ee061677ee1322095a3a30bd60a050170bdd32\": container with ID starting with 0d35eaebd2222b9a8d7695dcd8ee061677ee1322095a3a30bd60a050170bdd32 not found: ID does not exist" Aug 04 10:23:35.325476 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:35.325451 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx"] Aug 04 10:23:35.332566 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:35.332537 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-9bae9cd1-openshift-default-7c69765744-f25vx"] Aug 04 10:23:35.779120 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:35.779087 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62a12e28-53aa-4563-95d4-a428c8dd9b4f" path="/var/lib/kubelet/pods/62a12e28-53aa-4563-95d4-a428c8dd9b4f/volumes" Aug 04 10:23:35.985765 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:35.985729 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:23:36.135031 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:36.134941 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:23:36.315235 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:36.315198 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:23:36.655890 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:36.655856 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:23:36.758130 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:23:36.758096 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:24:26.042626 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:24:26.042494 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:25:31.955384 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:25:31.955353 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sdsr_7858cfca-73db-4ee1-9882-c13444f112a5/ovn-acl-logging/0.log" Aug 04 10:25:31.962226 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:25:31.962203 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8sdsr_7858cfca-73db-4ee1-9882-c13444f112a5/ovn-acl-logging/0.log" Aug 04 10:25:36.125339 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:25:36.125302 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:27:48.523431 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:27:48.523382 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:27:48.665267 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:27:48.665224 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:27:48.801685 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:27:48.801575 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:27:49.108225 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:27:49.108085 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:27:49.458485 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:27:49.458450 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:27:55.975494 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:27:55.975458 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-cx42r_526d3ebd-1f9f-49bc-8fcb-08a3ee209794/manager/0.log" Aug 04 10:27:56.116864 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:27:56.116824 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-6jrrz_7c2413b7-aca2-40ae-b13a-b4d193968c4a/manager/0.log" Aug 04 10:27:56.406567 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:27:56.406467 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-fxnjm_bad016c6-1c85-4855-8b2e-a9772ecd302a/server/0.log" Aug 04 10:27:56.529676 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:27:56.529647 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-n28ms_8dfa4811-aea8-4339-b5f1-d288941e3186/manager/0.log" Aug 04 10:27:56.947465 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:27:56.947438 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bb649b865-shttb_a6865d23-1fea-4c03-87d9-b6cba1b0cebe/copy-manifests/0.log" Aug 04 10:27:56.968239 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:27:56.968196 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bb649b865-shttb_a6865d23-1fea-4c03-87d9-b6cba1b0cebe/manager/0.log" Aug 04 10:27:58.667936 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:27:58.667903 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-7shjd_29bf9ef7-f611-4f5b-a7d2-f4054e519d00/manager/0.log" Aug 04 10:27:58.792848 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:27:58.792791 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-89kk7_79788882-8745-4be1-9ede-95fa3bf45a3d/manager/0.log" Aug 04 10:27:58.907922 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:27:58.907894 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-9c4tp_b8b566f6-96fd-493e-ad07-1eb86d960581/kuadrant-console-plugin/0.log" Aug 04 10:27:59.032432 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:27:59.032405 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-68cf589589-8sk9p_1b562b77-39d4-4f31-a447-b6a5d23e8a31/manager/0.log" Aug 04 10:27:59.299771 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:27:59.299691 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-fjlxt_e75bf807-5958-4c94-8f34-69611f539dc8/manager/0.log" Aug 04 10:27:59.676283 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:27:59.676202 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc_2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba/istio-proxy/0.log" Aug 04 10:27:59.918767 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:27:59.918741 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b474cb6fc-b764w_9ec790de-ee7b-451e-be4c-82fc33248f9b/kube-auth-proxy/0.log" Aug 04 10:28:00.178098 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:00.178069 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6284q_1c84a000-d9ad-433f-881b-5cb6e5b32088/istio-proxy/1.log" Aug 04 10:28:00.444833 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:00.444724 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-gj8d6_92d63f31-9830-471b-9e5b-2707def24198/payload-processing/0.log" Aug 04 10:28:00.559858 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:00.559825 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-69f7b748fb-vt9zm_ee459213-237c-404d-aa9d-8f681ee6e595/router/0.log" Aug 04 10:28:01.043353 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:01.043325 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-79hxw_bb85e7e2-bb03-4ea3-b44b-1cf5091c1ac2/main/0.log" Aug 04 10:28:01.415703 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:01.415624 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-gk4wm_5121391e-8a7a-4897-8bf2-1a8c82d5632b/main/0.log" Aug 04 10:28:01.530217 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:01.530185 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-pr9jv_28d347db-4831-422b-9ded-03103b007983/main/0.log" Aug 04 10:28:07.103316 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:07.103278 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:08.383614 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:08.383548 2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" limit=1024 Aug 04 10:28:10.051283 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:10.051248 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7wl64_8f732a38-dafb-42d2-acc6-ff8059c65350/global-pull-secret-syncer/0.log" Aug 04 10:28:10.221458 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:10.221429 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hgcgb_781f670f-6934-4504-810d-95429f368c7f/konnectivity-agent/0.log" Aug 04 10:28:10.357621 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:10.357543 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-241.ec2.internal_88730f4d5c4cac33f5b85009ff17427c/haproxy/0.log" Aug 04 10:28:15.129493 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:15.129461 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-7shjd_29bf9ef7-f611-4f5b-a7d2-f4054e519d00/manager/0.log" Aug 04 10:28:15.167525 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:15.167498 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-89kk7_79788882-8745-4be1-9ede-95fa3bf45a3d/manager/0.log" Aug 04 10:28:15.202308 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:15.202282 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-9c4tp_b8b566f6-96fd-493e-ad07-1eb86d960581/kuadrant-console-plugin/0.log" Aug 04 10:28:15.329081 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:15.329048 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-68cf589589-8sk9p_1b562b77-39d4-4f31-a447-b6a5d23e8a31/manager/0.log" Aug 04 10:28:15.448465 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:15.448439 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-fjlxt_e75bf807-5958-4c94-8f34-69611f539dc8/manager/0.log" Aug 04 10:28:17.119981 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:17.119878 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ee3517f9-52ed-4ae1-aba4-5f3e3668b814/alertmanager/0.log" Aug 04 10:28:17.346195 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:17.346167 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ee3517f9-52ed-4ae1-aba4-5f3e3668b814/config-reloader/0.log" Aug 04 10:28:17.477868 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:17.477837 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ee3517f9-52ed-4ae1-aba4-5f3e3668b814/kube-rbac-proxy-web/0.log" Aug 04 10:28:17.659069 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:17.659045 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ee3517f9-52ed-4ae1-aba4-5f3e3668b814/kube-rbac-proxy/0.log" Aug 04 10:28:17.822138 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:17.822061 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ee3517f9-52ed-4ae1-aba4-5f3e3668b814/kube-rbac-proxy-metric/0.log" Aug 04 10:28:17.904958 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:17.904924 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ee3517f9-52ed-4ae1-aba4-5f3e3668b814/prom-label-proxy/0.log" Aug 04 10:28:17.969531 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:17.969502 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ee3517f9-52ed-4ae1-aba4-5f3e3668b814/init-config-reloader/0.log" Aug 04 10:28:18.285217 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:18.285191 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-757d6d8489-6kbhk_102ce85d-672a-4526-bdae-be62d5d43c4c/metrics-server/0.log" Aug 04 10:28:18.647012 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:18.646938 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zctwh_c6322d16-7b64-41ee-8ff2-dbb858a7bc96/node-exporter/0.log" Aug 04 10:28:18.680560 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:18.680536 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zctwh_c6322d16-7b64-41ee-8ff2-dbb858a7bc96/kube-rbac-proxy/0.log" Aug 04 10:28:18.730182 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:18.730152 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zctwh_c6322d16-7b64-41ee-8ff2-dbb858a7bc96/init-textfile/0.log" Aug 04 10:28:19.221761 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.221729 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-758d7cb874-zlccm_190841b0-afa9-47c0-92ed-b4eb283ba6d5/prometheus-operator/0.log" Aug 04 10:28:19.271176 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.271148 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-758d7cb874-zlccm_190841b0-afa9-47c0-92ed-b4eb283ba6d5/kube-rbac-proxy/0.log" Aug 04 10:28:19.277849 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.277817 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-q6f76/perf-node-gather-daemonset-tvt8h"] Aug 04 10:28:19.278259 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.278246 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62a12e28-53aa-4563-95d4-a428c8dd9b4f" containerName="istio-proxy" Aug 04 10:28:19.278309 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.278260 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="62a12e28-53aa-4563-95d4-a428c8dd9b4f" containerName="istio-proxy" Aug 04 10:28:19.278344 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.278336 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="62a12e28-53aa-4563-95d4-a428c8dd9b4f" containerName="istio-proxy" Aug 04 10:28:19.281489 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.281470 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-tvt8h" Aug 04 10:28:19.284489 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.284465 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-q6f76\"/\"default-dockercfg-fm545\"" Aug 04 10:28:19.285478 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.285456 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-q6f76\"/\"kube-root-ca.crt\"" Aug 04 10:28:19.285859 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.285707 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-q6f76\"/\"openshift-service-ca.crt\"" Aug 04 10:28:19.302521 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.302495 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q6f76/perf-node-gather-daemonset-tvt8h"] Aug 04 10:28:19.383812 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.383769 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/54da13d4-1e31-4f8b-8709-cac8b0f33c35-proc\") pod \"perf-node-gather-daemonset-tvt8h\" (UID: \"54da13d4-1e31-4f8b-8709-cac8b0f33c35\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-tvt8h" Aug 04 10:28:19.384033 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.383841 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/54da13d4-1e31-4f8b-8709-cac8b0f33c35-podres\") pod \"perf-node-gather-daemonset-tvt8h\" (UID: \"54da13d4-1e31-4f8b-8709-cac8b0f33c35\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-tvt8h" Aug 04 10:28:19.384033 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.383883 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54da13d4-1e31-4f8b-8709-cac8b0f33c35-sys\") pod \"perf-node-gather-daemonset-tvt8h\" (UID: \"54da13d4-1e31-4f8b-8709-cac8b0f33c35\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-tvt8h" Aug 04 10:28:19.384033 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.383933 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/54da13d4-1e31-4f8b-8709-cac8b0f33c35-lib-modules\") pod \"perf-node-gather-daemonset-tvt8h\" (UID: \"54da13d4-1e31-4f8b-8709-cac8b0f33c35\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-tvt8h" Aug 04 10:28:19.384033 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.383964 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xb8fc\" (UniqueName: \"kubernetes.io/projected/54da13d4-1e31-4f8b-8709-cac8b0f33c35-kube-api-access-xb8fc\") pod \"perf-node-gather-daemonset-tvt8h\" (UID: \"54da13d4-1e31-4f8b-8709-cac8b0f33c35\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-tvt8h" Aug 04 10:28:19.402610 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.402588 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c8459789c-glxzj_d404891b-cd94-4e3d-8955-3734c1265f34/telemeter-client/0.log" Aug 04 10:28:19.446743 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.446716 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c8459789c-glxzj_d404891b-cd94-4e3d-8955-3734c1265f34/reload/0.log" Aug 04 10:28:19.480793 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.480707 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c8459789c-glxzj_d404891b-cd94-4e3d-8955-3734c1265f34/kube-rbac-proxy/0.log" Aug 04 10:28:19.485061 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.485039 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/54da13d4-1e31-4f8b-8709-cac8b0f33c35-lib-modules\") pod \"perf-node-gather-daemonset-tvt8h\" (UID: \"54da13d4-1e31-4f8b-8709-cac8b0f33c35\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-tvt8h" Aug 04 10:28:19.485145 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.485072 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb8fc\" (UniqueName: \"kubernetes.io/projected/54da13d4-1e31-4f8b-8709-cac8b0f33c35-kube-api-access-xb8fc\") pod \"perf-node-gather-daemonset-tvt8h\" (UID: \"54da13d4-1e31-4f8b-8709-cac8b0f33c35\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-tvt8h" Aug 04 10:28:19.485193 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.485144 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/54da13d4-1e31-4f8b-8709-cac8b0f33c35-proc\") pod \"perf-node-gather-daemonset-tvt8h\" (UID: \"54da13d4-1e31-4f8b-8709-cac8b0f33c35\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-tvt8h" Aug 04 10:28:19.485193 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.485160 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/54da13d4-1e31-4f8b-8709-cac8b0f33c35-podres\") pod \"perf-node-gather-daemonset-tvt8h\" (UID: \"54da13d4-1e31-4f8b-8709-cac8b0f33c35\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-tvt8h" Aug 04 10:28:19.485275 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.485191 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54da13d4-1e31-4f8b-8709-cac8b0f33c35-sys\") pod \"perf-node-gather-daemonset-tvt8h\" (UID: \"54da13d4-1e31-4f8b-8709-cac8b0f33c35\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-tvt8h" Aug 04 10:28:19.485275 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.485238 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/54da13d4-1e31-4f8b-8709-cac8b0f33c35-lib-modules\") pod \"perf-node-gather-daemonset-tvt8h\" (UID: \"54da13d4-1e31-4f8b-8709-cac8b0f33c35\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-tvt8h" Aug 04 10:28:19.485275 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.485242 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/54da13d4-1e31-4f8b-8709-cac8b0f33c35-proc\") pod \"perf-node-gather-daemonset-tvt8h\" (UID: \"54da13d4-1e31-4f8b-8709-cac8b0f33c35\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-tvt8h" Aug 04 10:28:19.485376 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.485277 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54da13d4-1e31-4f8b-8709-cac8b0f33c35-sys\") pod \"perf-node-gather-daemonset-tvt8h\" (UID: \"54da13d4-1e31-4f8b-8709-cac8b0f33c35\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-tvt8h" Aug 04 10:28:19.485376 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.485277 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/54da13d4-1e31-4f8b-8709-cac8b0f33c35-podres\") pod \"perf-node-gather-daemonset-tvt8h\" (UID: \"54da13d4-1e31-4f8b-8709-cac8b0f33c35\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-tvt8h" Aug 04 10:28:19.498301 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.498272 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xb8fc\" (UniqueName: \"kubernetes.io/projected/54da13d4-1e31-4f8b-8709-cac8b0f33c35-kube-api-access-xb8fc\") pod \"perf-node-gather-daemonset-tvt8h\" (UID: \"54da13d4-1e31-4f8b-8709-cac8b0f33c35\") " pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-tvt8h" Aug 04 10:28:19.591556 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.591521 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-tvt8h" Aug 04 10:28:19.736472 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.736440 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q6f76/perf-node-gather-daemonset-tvt8h"] Aug 04 10:28:19.736882 ip-10-0-129-241 kubenswrapper[2572]: W0804 10:28:19.736860 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod54da13d4_1e31_4f8b_8709_cac8b0f33c35.slice/crio-f679b51359efa458bf594638a0e9790f991565e5a44dedad032ddc8ce0d89f5d WatchSource:0}: Error finding container f679b51359efa458bf594638a0e9790f991565e5a44dedad032ddc8ce0d89f5d: Status 404 returned error can't find the container with id f679b51359efa458bf594638a0e9790f991565e5a44dedad032ddc8ce0d89f5d Aug 04 10:28:19.895632 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.895592 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-tvt8h" event={"ID":"54da13d4-1e31-4f8b-8709-cac8b0f33c35","Type":"ContainerStarted","Data":"2373bf3c5d3208c6d8eef49f0cd0545fecab1f12df6ae68f4fd677779e8a968c"} Aug 04 10:28:19.895833 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.895640 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-tvt8h" event={"ID":"54da13d4-1e31-4f8b-8709-cac8b0f33c35","Type":"ContainerStarted","Data":"f679b51359efa458bf594638a0e9790f991565e5a44dedad032ddc8ce0d89f5d"} Aug 04 10:28:19.895833 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.895738 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-tvt8h" Aug 04 10:28:19.922217 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:19.922160 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-tvt8h" podStartSLOduration=0.922144011 podStartE2EDuration="922.144011ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 10:28:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 10:28:19.919955256 +0000 UTC m=+2868.664089673" watchObservedRunningTime="2026-08-04 10:28:19.922144011 +0000 UTC m=+2868.666278430" Aug 04 10:28:22.013459 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:22.013429 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-9d7445797-6m6t7_3099ba6e-e9c2-4a88-a4ae-00b4a12ee7fc/console/0.log" Aug 04 10:28:22.065583 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:22.065555 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-77c974ffcf-bg556_8304a19e-b591-4a42-aecb-13e8bc84f5b9/download-server/0.log" Aug 04 10:28:22.734893 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:22.734865 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5b8776669-kptzz_ef014102-c117-4458-8b5c-169ae01b6d70/volume-data-source-validator/0.log" Aug 04 10:28:24.098342 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:24.098267 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cmck8_5cce2515-bbb6-4fc5-b6ee-9825796843f2/dns/0.log" Aug 04 10:28:24.140544 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:24.140517 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cmck8_5cce2515-bbb6-4fc5-b6ee-9825796843f2/kube-rbac-proxy/0.log" Aug 04 10:28:24.281086 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:24.281057 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qvwd8_fa0ec5ee-8ecd-4f22-ae99-cfbe2855395c/dns-node-resolver/0.log" Aug 04 10:28:25.027121 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:25.027089 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-588c6c96bb-gz8pg_7f40804f-b0dd-4a82-982c-da8591f7fdf5/registry/0.log" Aug 04 10:28:25.079439 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:25.079407 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-57rd7_a9ba7a09-0ff2-4657-8bbd-47b0af96a45a/node-ca/0.log" Aug 04 10:28:25.911081 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:25.911052 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-q6f76/perf-node-gather-daemonset-tvt8h" Aug 04 10:28:26.400279 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:26.400251 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc_2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba/istio-proxy/0.log" Aug 04 10:28:26.548970 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:26.548941 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b474cb6fc-b764w_9ec790de-ee7b-451e-be4c-82fc33248f9b/kube-auth-proxy/0.log" Aug 04 10:28:26.676771 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:26.676684 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6284q_1c84a000-d9ad-433f-881b-5cb6e5b32088/istio-proxy/0.log" Aug 04 10:28:26.683274 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:26.683253 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6284q_1c84a000-d9ad-433f-881b-5cb6e5b32088/istio-proxy/1.log" Aug 04 10:28:26.795844 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:26.795815 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-gj8d6_92d63f31-9830-471b-9e5b-2707def24198/payload-processing/0.log" Aug 04 10:28:26.845329 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:26.845300 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-69f7b748fb-vt9zm_ee459213-237c-404d-aa9d-8f681ee6e595/router/0.log" Aug 04 10:28:27.811181 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:27.811152 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z68g5_3d4ecc98-460d-41bb-9fb6-fcea7a43d789/serve-healthcheck-canary/0.log" Aug 04 10:28:28.690059 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:28.689980 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nn7rk_11e66e79-a3c2-4c7c-a283-0fc4ecffd574/kube-rbac-proxy/0.log" Aug 04 10:28:28.718290 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:28.718256 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nn7rk_11e66e79-a3c2-4c7c-a283-0fc4ecffd574/exporter/0.log" Aug 04 10:28:28.764317 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:28.764282 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nn7rk_11e66e79-a3c2-4c7c-a283-0fc4ecffd574/extractor/0.log" Aug 04 10:28:30.076394 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:30.076367 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc_2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba/istio-proxy/0.log" Aug 04 10:28:30.153298 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:30.153272 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b474cb6fc-b764w_9ec790de-ee7b-451e-be4c-82fc33248f9b/kube-auth-proxy/0.log" Aug 04 10:28:30.195068 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:30.195034 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6284q_1c84a000-d9ad-433f-881b-5cb6e5b32088/istio-proxy/0.log" Aug 04 10:28:30.201479 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:30.201457 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6284q_1c84a000-d9ad-433f-881b-5cb6e5b32088/istio-proxy/1.log" Aug 04 10:28:30.251751 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:30.251725 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-gj8d6_92d63f31-9830-471b-9e5b-2707def24198/payload-processing/0.log" Aug 04 10:28:30.276299 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:30.276273 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-69f7b748fb-vt9zm_ee459213-237c-404d-aa9d-8f681ee6e595/router/0.log" Aug 04 10:28:31.294482 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:31.294448 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-cx42r_526d3ebd-1f9f-49bc-8fcb-08a3ee209794/manager/0.log" Aug 04 10:28:31.387831 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:31.387787 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-nh8gc_2e0c7407-0a0a-4791-8b8c-f3d2609ad8ba/istio-proxy/0.log" Aug 04 10:28:31.424580 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:31.424550 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-6jrrz_7c2413b7-aca2-40ae-b13a-b4d193968c4a/manager/0.log" Aug 04 10:28:31.504956 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:31.504926 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b474cb6fc-b764w_9ec790de-ee7b-451e-be4c-82fc33248f9b/kube-auth-proxy/0.log" Aug 04 10:28:31.645589 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:31.645504 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6284q_1c84a000-d9ad-433f-881b-5cb6e5b32088/istio-proxy/0.log" Aug 04 10:28:31.648746 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:31.648721 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-6284q_1c84a000-d9ad-433f-881b-5cb6e5b32088/istio-proxy/1.log" Aug 04 10:28:31.688141 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:31.688109 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-fxnjm_bad016c6-1c85-4855-8b2e-a9772ecd302a/server/0.log" Aug 04 10:28:31.721179 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:31.721145 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-gj8d6_92d63f31-9830-471b-9e5b-2707def24198/payload-processing/0.log" Aug 04 10:28:31.736993 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:31.736967 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-69f7b748fb-vt9zm_ee459213-237c-404d-aa9d-8f681ee6e595/router/0.log" Aug 04 10:28:31.755413 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:31.755382 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-n28ms_8dfa4811-aea8-4339-b5f1-d288941e3186/manager/0.log" Aug 04 10:28:32.003331 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:32.003300 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bb649b865-shttb_a6865d23-1fea-4c03-87d9-b6cba1b0cebe/manager/0.log" Aug 04 10:28:32.027368 ip-10-0-129-241 kubenswrapper[2572]: I0804 10:28:32.027332 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bb649b865-shttb_a6865d23-1fea-4c03-87d9-b6cba1b0cebe/copy-manifests/0.log"