Jul 06 23:48:36.515362 ip-10-0-136-25 systemd[1]: Starting Kubernetes Kubelet... Jul 06 23:48:36.861487 ip-10-0-136-25 kubenswrapper[2553]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 23:48:36.861487 ip-10-0-136-25 kubenswrapper[2553]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 23:48:36.861487 ip-10-0-136-25 kubenswrapper[2553]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 23:48:36.861487 ip-10-0-136-25 kubenswrapper[2553]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 23:48:36.861487 ip-10-0-136-25 kubenswrapper[2553]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 23:48:36.861487 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.840880 2553 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 23:48:36.861487 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844254 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:48:36.861487 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844265 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:48:36.861487 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844268 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:48:36.861487 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844271 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:48:36.861487 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844274 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:48:36.861487 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844279 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:48:36.861487 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844282 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:48:36.861487 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844285 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:48:36.862792 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844288 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:48:36.862792 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844290 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:48:36.862792 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844293 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:48:36.862792 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844295 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:48:36.862792 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844298 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:48:36.862792 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844300 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:48:36.862792 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844303 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:48:36.862792 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844306 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:48:36.862792 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844308 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:48:36.862792 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844311 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:48:36.862792 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844313 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:48:36.862792 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844317 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:48:36.862792 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844320 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:48:36.862792 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844323 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:48:36.862792 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844326 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:48:36.862792 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844329 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:48:36.862792 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844332 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:48:36.862792 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844335 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:48:36.862792 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844337 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:48:36.862792 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844340 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:48:36.863650 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844343 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:48:36.863650 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844345 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:48:36.863650 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844348 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:48:36.863650 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844351 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:48:36.863650 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844353 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:48:36.863650 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844356 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:48:36.863650 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844358 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:48:36.863650 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844361 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:48:36.863650 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844363 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:48:36.863650 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844373 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:48:36.863650 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844375 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:48:36.863650 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844379 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:48:36.863650 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844383 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:48:36.863650 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844386 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:48:36.863650 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844388 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:48:36.863650 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844391 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:48:36.863650 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844394 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:48:36.863650 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844396 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:48:36.863650 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844398 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:48:36.864530 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844430 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:48:36.864530 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844433 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:48:36.864530 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844450 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:48:36.864530 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844453 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:48:36.864530 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844456 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:48:36.864530 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844459 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:48:36.864530 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844462 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:48:36.864530 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844464 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:48:36.864530 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844468 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:48:36.864530 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844471 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:48:36.864530 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844473 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:48:36.864530 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844476 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:48:36.864530 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844478 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:48:36.864530 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844481 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:48:36.864530 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844483 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:48:36.864530 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844486 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:48:36.864530 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844488 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:48:36.864530 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844491 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:48:36.864530 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844493 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:48:36.864530 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844496 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:48:36.865410 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844498 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:48:36.865410 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844501 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:48:36.865410 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844503 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:48:36.865410 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844505 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:48:36.865410 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844509 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:48:36.865410 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844515 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:48:36.865410 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844518 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:48:36.865410 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844521 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:48:36.865410 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844525 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:48:36.865410 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844528 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:48:36.865410 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844531 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:48:36.865410 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844533 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:48:36.865410 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844536 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:48:36.865410 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844539 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:48:36.865410 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844542 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:48:36.865410 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844550 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:48:36.865410 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844553 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:48:36.865410 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844556 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:48:36.865410 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844559 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:48:36.866300 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844562 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:48:36.866300 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844565 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:48:36.866300 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844567 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:48:36.866300 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844570 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:48:36.866300 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844572 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:48:36.866300 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844575 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:48:36.866300 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844577 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:48:36.866300 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844581 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:48:36.866300 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844584 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:48:36.866300 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844587 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:48:36.866300 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844590 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:48:36.866300 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844592 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:48:36.866300 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.844595 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:48:36.866300 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845391 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:48:36.866300 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845398 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:48:36.866300 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845400 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:48:36.866300 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845403 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:48:36.866300 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845406 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:48:36.866300 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845409 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:48:36.867381 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845411 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:48:36.867381 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845414 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:48:36.867381 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845417 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:48:36.867381 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845420 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:48:36.867381 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845422 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:48:36.867381 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845425 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:48:36.867381 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845428 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:48:36.867381 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845430 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:48:36.867381 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845436 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:48:36.867381 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845438 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:48:36.867381 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845448 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:48:36.867381 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845450 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:48:36.867381 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845453 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:48:36.867381 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845456 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:48:36.867381 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845458 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:48:36.867381 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845461 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:48:36.867381 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845463 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:48:36.867381 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845466 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:48:36.867381 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845469 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:48:36.867381 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845471 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:48:36.868276 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845473 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:48:36.868276 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845476 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:48:36.868276 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845478 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:48:36.868276 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845481 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:48:36.868276 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845484 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:48:36.868276 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845487 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:48:36.868276 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845489 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:48:36.868276 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845492 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:48:36.868276 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845494 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:48:36.868276 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845497 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:48:36.868276 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845502 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:48:36.868276 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845505 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:48:36.868276 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845508 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:48:36.868276 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845512 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:48:36.868276 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845516 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:48:36.868276 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845519 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:48:36.868276 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845522 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:48:36.868276 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845524 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:48:36.868276 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845527 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:48:36.869163 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845529 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:48:36.869163 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845538 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:48:36.869163 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845541 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:48:36.869163 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845543 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:48:36.869163 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845551 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:48:36.869163 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845554 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:48:36.869163 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845557 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:48:36.869163 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845559 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:48:36.869163 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845562 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:48:36.869163 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845564 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:48:36.869163 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845566 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:48:36.869163 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845569 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:48:36.869163 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845572 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:48:36.869163 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845574 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:48:36.869163 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845577 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:48:36.869163 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845579 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:48:36.869163 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845582 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:48:36.869163 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845584 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:48:36.869163 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845587 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:48:36.870024 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845589 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:48:36.870024 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845592 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:48:36.870024 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845596 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:48:36.870024 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845599 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:48:36.870024 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845602 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:48:36.870024 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845605 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:48:36.870024 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845607 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:48:36.870024 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845609 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:48:36.870024 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845613 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:48:36.870024 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845615 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:48:36.870024 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845618 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:48:36.870024 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845620 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:48:36.870024 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845623 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:48:36.870024 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845626 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:48:36.870024 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845629 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:48:36.870024 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845632 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:48:36.870024 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845634 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:48:36.870024 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845637 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:48:36.870024 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845645 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:48:36.870024 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845647 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:48:36.870893 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845650 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:48:36.870893 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845652 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:48:36.870893 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845655 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:48:36.870893 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845657 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:48:36.870893 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845660 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:48:36.870893 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845662 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:48:36.870893 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845665 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:48:36.870893 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845668 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:48:36.870893 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845670 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:48:36.870893 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845673 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:48:36.870893 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845675 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:48:36.870893 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845678 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:48:36.870893 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845680 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:48:36.870893 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845683 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:48:36.870893 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.845685 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:48:36.870893 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846179 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:48:36.870893 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846186 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:48:36.870893 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846189 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:48:36.870893 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846193 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:48:36.870893 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846196 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:48:36.871761 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846199 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:48:36.871761 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846201 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:48:36.871761 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846204 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:48:36.871761 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846207 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:48:36.871761 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846209 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:48:36.871761 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846212 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:48:36.871761 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846215 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:48:36.871761 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846218 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:48:36.871761 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846220 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:48:36.871761 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846223 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:48:36.871761 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846231 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:48:36.871761 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846234 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:48:36.871761 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846236 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:48:36.871761 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846239 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:48:36.871761 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846241 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:48:36.871761 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846244 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:48:36.871761 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846247 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:48:36.871761 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846249 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:48:36.871761 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846252 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:48:36.871761 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846255 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:48:36.872627 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846258 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:48:36.872627 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846260 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:48:36.872627 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846263 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:48:36.872627 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846265 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:48:36.872627 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846268 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:48:36.872627 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846272 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:48:36.872627 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846275 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:48:36.872627 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846277 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:48:36.872627 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846280 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:48:36.872627 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846283 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:48:36.872627 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846285 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:48:36.872627 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846288 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:48:36.872627 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846290 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:48:36.872627 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846293 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:48:36.872627 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846295 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:48:36.872627 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846298 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:48:36.872627 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846300 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:48:36.872627 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846303 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:48:36.872627 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846305 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:48:36.872627 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846308 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:48:36.873495 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846310 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:48:36.873495 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846313 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:48:36.873495 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846316 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:48:36.873495 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846673 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:48:36.873495 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846676 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:48:36.873495 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846679 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:48:36.873495 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846682 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:48:36.873495 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846685 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:48:36.873495 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846687 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:48:36.873495 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846690 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:48:36.873495 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846693 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:48:36.873495 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846695 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:48:36.873495 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846698 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:48:36.873495 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846701 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:48:36.873495 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846703 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:48:36.873495 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846706 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:48:36.873495 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846708 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:48:36.873495 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846712 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:48:36.873495 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846715 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:48:36.874382 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846717 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:48:36.874382 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846719 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:48:36.874382 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846722 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:48:36.874382 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846724 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:48:36.874382 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846727 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:48:36.874382 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846731 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:48:36.874382 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846734 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:48:36.874382 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846737 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:48:36.874382 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846739 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:48:36.874382 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846743 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:48:36.874382 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846746 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:48:36.874382 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846749 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:48:36.874382 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846751 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:48:36.874382 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846754 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:48:36.874382 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846756 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:48:36.874382 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846759 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:48:36.874382 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846761 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:48:36.874382 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846764 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:48:36.874382 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846767 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:48:36.874382 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846770 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:48:36.875250 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846772 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:48:36.875250 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846776 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:48:36.875250 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846779 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:48:36.875250 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846782 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:48:36.875250 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846786 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:48:36.875250 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846789 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:48:36.875250 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846791 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:48:36.875250 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846794 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:48:36.875250 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846796 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:48:36.875250 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846800 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:48:36.875250 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846804 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:48:36.875250 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846807 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:48:36.875250 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846810 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:48:36.875250 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846812 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:48:36.875250 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.846815 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:48:36.875250 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.846876 2553 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 23:48:36.875250 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.846888 2553 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 23:48:36.875250 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.846894 2553 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 23:48:36.875250 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.846899 2553 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 23:48:36.875250 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.846903 2553 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 23:48:36.876141 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.846907 2553 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 23:48:36.876141 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.846911 2553 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 23:48:36.876141 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.846915 2553 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 23:48:36.876141 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.846918 2553 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 23:48:36.876141 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.846921 2553 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 23:48:36.876141 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.846924 2553 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 23:48:36.876141 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.846928 2553 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 23:48:36.876141 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.846931 2553 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 23:48:36.876141 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.846934 2553 flags.go:64] FLAG: --cgroup-root="" Jul 06 23:48:36.876141 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.846937 2553 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 23:48:36.876141 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.846940 2553 flags.go:64] FLAG: --client-ca-file="" Jul 06 23:48:36.876141 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.846943 2553 flags.go:64] FLAG: --cloud-provider="external" Jul 06 23:48:36.876141 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.846946 2553 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 23:48:36.876141 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.846963 2553 flags.go:64] FLAG: --cluster-domain="" Jul 06 23:48:36.876141 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.846967 2553 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 23:48:36.876141 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.846970 2553 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 23:48:36.876141 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.846974 2553 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 23:48:36.876141 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.846977 2553 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 23:48:36.876141 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.846981 2553 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 23:48:36.876141 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.846984 2553 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 23:48:36.876141 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.846987 2553 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 23:48:36.876141 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.846990 2553 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 23:48:36.876141 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.846994 2553 flags.go:64] FLAG: --contention-profiling="false" Jul 06 23:48:36.876141 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.846997 2553 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 23:48:36.877418 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847001 2553 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 23:48:36.877418 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847004 2553 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 23:48:36.877418 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847006 2553 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 23:48:36.877418 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847010 2553 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 23:48:36.877418 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847013 2553 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 23:48:36.877418 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847016 2553 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 23:48:36.877418 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847019 2553 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 23:48:36.877418 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847022 2553 flags.go:64] FLAG: --enable-server="true" Jul 06 23:48:36.877418 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847025 2553 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 23:48:36.877418 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847029 2553 flags.go:64] FLAG: --event-burst="100" Jul 06 23:48:36.877418 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847032 2553 flags.go:64] FLAG: --event-qps="50" Jul 06 23:48:36.877418 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847035 2553 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 23:48:36.877418 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847038 2553 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 23:48:36.877418 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847041 2553 flags.go:64] FLAG: --eviction-hard="" Jul 06 23:48:36.877418 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847044 2553 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 23:48:36.877418 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847047 2553 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 23:48:36.877418 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847050 2553 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 23:48:36.877418 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847053 2553 flags.go:64] FLAG: --eviction-soft="" Jul 06 23:48:36.877418 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847056 2553 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 23:48:36.877418 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847059 2553 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 23:48:36.877418 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847062 2553 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 23:48:36.877418 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847066 2553 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 23:48:36.877418 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847069 2553 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 23:48:36.877418 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847072 2553 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 23:48:36.877418 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847074 2553 flags.go:64] FLAG: --feature-gates="" Jul 06 23:48:36.878527 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847078 2553 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 23:48:36.878527 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847081 2553 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 23:48:36.878527 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847084 2553 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 23:48:36.878527 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847087 2553 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 23:48:36.878527 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847090 2553 flags.go:64] FLAG: --healthz-port="10248" Jul 06 23:48:36.878527 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847094 2553 flags.go:64] FLAG: --help="false" Jul 06 23:48:36.878527 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847097 2553 flags.go:64] FLAG: --hostname-override="ip-10-0-136-25.ec2.internal" Jul 06 23:48:36.878527 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847100 2553 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 23:48:36.878527 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847103 2553 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 23:48:36.878527 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847106 2553 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 23:48:36.878527 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847109 2553 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 23:48:36.878527 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847113 2553 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 23:48:36.878527 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847129 2553 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 23:48:36.878527 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847132 2553 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 23:48:36.878527 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847135 2553 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 23:48:36.878527 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847138 2553 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 23:48:36.878527 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847141 2553 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 23:48:36.878527 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847144 2553 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 23:48:36.878527 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847147 2553 flags.go:64] FLAG: --kube-reserved="" Jul 06 23:48:36.878527 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847150 2553 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 23:48:36.878527 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847152 2553 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 23:48:36.878527 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847155 2553 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 23:48:36.878527 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847158 2553 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 23:48:36.878527 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847161 2553 flags.go:64] FLAG: --lock-file="" Jul 06 23:48:36.879610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847164 2553 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 23:48:36.879610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847167 2553 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 23:48:36.879610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847170 2553 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 23:48:36.879610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847175 2553 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 23:48:36.879610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847179 2553 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 23:48:36.879610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847182 2553 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 23:48:36.879610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847185 2553 flags.go:64] FLAG: --logging-format="text" Jul 06 23:48:36.879610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847188 2553 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 23:48:36.879610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847192 2553 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 23:48:36.879610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847194 2553 flags.go:64] FLAG: --manifest-url="" Jul 06 23:48:36.879610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847197 2553 flags.go:64] FLAG: --manifest-url-header="" Jul 06 23:48:36.879610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847204 2553 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 23:48:36.879610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847207 2553 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 23:48:36.879610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847213 2553 flags.go:64] FLAG: --max-pods="110" Jul 06 23:48:36.879610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847216 2553 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 23:48:36.879610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847219 2553 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 23:48:36.879610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847222 2553 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 23:48:36.879610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847225 2553 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 23:48:36.879610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847228 2553 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 23:48:36.879610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847230 2553 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 23:48:36.879610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847233 2553 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 23:48:36.879610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847240 2553 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 23:48:36.879610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847244 2553 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 23:48:36.879610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847247 2553 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 23:48:36.879610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847250 2553 flags.go:64] FLAG: --pod-cidr="" Jul 06 23:48:36.882851 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847252 2553 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 23:48:36.882851 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847258 2553 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 23:48:36.882851 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847261 2553 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 23:48:36.882851 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847264 2553 flags.go:64] FLAG: --pods-per-core="0" Jul 06 23:48:36.882851 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847267 2553 flags.go:64] FLAG: --port="10250" Jul 06 23:48:36.882851 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847270 2553 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 23:48:36.882851 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847273 2553 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0da6b5ff9aed53b79" Jul 06 23:48:36.882851 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847276 2553 flags.go:64] FLAG: --qos-reserved="" Jul 06 23:48:36.882851 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847278 2553 flags.go:64] FLAG: --read-only-port="10255" Jul 06 23:48:36.882851 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847281 2553 flags.go:64] FLAG: --register-node="true" Jul 06 23:48:36.882851 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847284 2553 flags.go:64] FLAG: --register-with-taints="" Jul 06 23:48:36.882851 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847289 2553 flags.go:64] FLAG: --registry-burst="10" Jul 06 23:48:36.882851 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847292 2553 flags.go:64] FLAG: --registry-qps="5" Jul 06 23:48:36.882851 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847295 2553 flags.go:64] FLAG: --reserved-cpus="" Jul 06 23:48:36.882851 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847298 2553 flags.go:64] FLAG: --reserved-memory="" Jul 06 23:48:36.882851 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847301 2553 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 23:48:36.882851 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847304 2553 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 23:48:36.882851 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847307 2553 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 23:48:36.882851 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847310 2553 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 23:48:36.882851 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847313 2553 flags.go:64] FLAG: --runonce="false" Jul 06 23:48:36.882851 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847316 2553 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 23:48:36.882851 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847320 2553 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 23:48:36.882851 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847323 2553 flags.go:64] FLAG: --seccomp-default="false" Jul 06 23:48:36.882851 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847326 2553 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 23:48:36.882851 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847329 2553 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 23:48:36.973353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847332 2553 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 23:48:36.973353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847335 2553 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 23:48:36.973353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847338 2553 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 23:48:36.973353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847341 2553 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 23:48:36.973353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847344 2553 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 23:48:36.973353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847347 2553 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 23:48:36.973353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847349 2553 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 23:48:36.973353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847352 2553 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 23:48:36.973353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847355 2553 flags.go:64] FLAG: --system-cgroups="" Jul 06 23:48:36.973353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847358 2553 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 23:48:36.973353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847363 2553 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 23:48:36.973353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847365 2553 flags.go:64] FLAG: --tls-cert-file="" Jul 06 23:48:36.973353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847368 2553 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 23:48:36.973353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847371 2553 flags.go:64] FLAG: --tls-min-version="" Jul 06 23:48:36.973353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847374 2553 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 23:48:36.973353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847377 2553 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 23:48:36.973353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847380 2553 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 23:48:36.973353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847382 2553 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 23:48:36.973353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847387 2553 flags.go:64] FLAG: --v="2" Jul 06 23:48:36.973353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847391 2553 flags.go:64] FLAG: --version="false" Jul 06 23:48:36.973353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847395 2553 flags.go:64] FLAG: --vmodule="" Jul 06 23:48:36.973353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847398 2553 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 23:48:36.973353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847402 2553 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 23:48:36.973353 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847516 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:48:36.943293 ip-10-0-136-25 systemd[1]: Started Kubernetes Kubelet. Jul 06 23:48:36.980877 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847520 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:48:36.980877 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847523 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:48:36.980877 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847526 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:48:36.980877 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847529 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:48:36.980877 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847534 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:48:36.980877 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847537 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:48:36.980877 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847540 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:48:36.980877 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847543 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:48:36.980877 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847546 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:48:36.980877 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847548 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:48:36.980877 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847552 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:48:36.980877 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847555 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:48:36.980877 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847558 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:48:36.980877 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847561 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:48:36.980877 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847564 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:48:36.980877 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847567 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:48:36.980877 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847570 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:48:36.980877 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847573 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:48:36.980877 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847575 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:48:36.981962 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847578 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:48:36.981962 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847581 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:48:36.981962 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847583 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:48:36.981962 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847586 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:48:36.981962 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847588 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:48:36.981962 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847591 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:48:36.981962 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847594 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:48:36.981962 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847599 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:48:36.981962 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847602 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:48:36.981962 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847605 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:48:36.981962 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847610 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:48:36.981962 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847615 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:48:36.981962 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847618 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:48:36.981962 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847621 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:48:36.981962 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847623 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:48:36.981962 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847626 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:48:36.981962 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847629 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:48:36.981962 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847632 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:48:36.981962 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847635 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:48:36.982832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847638 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:48:36.982832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847640 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:48:36.982832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847643 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:48:36.982832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847645 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:48:36.982832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847648 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:48:36.982832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847651 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:48:36.982832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847653 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:48:36.982832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847655 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:48:36.982832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847658 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:48:36.982832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847661 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:48:36.982832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847663 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:48:36.982832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847666 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:48:36.982832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847668 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:48:36.982832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847671 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:48:36.982832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847674 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:48:36.982832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847677 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:48:36.982832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847680 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:48:36.982832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847683 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:48:36.982832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847685 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:48:36.982832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847688 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:48:36.983928 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847691 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:48:36.983928 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847694 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:48:36.983928 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847696 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:48:36.983928 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847699 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:48:36.983928 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847702 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:48:36.983928 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847705 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:48:36.983928 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847707 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:48:36.983928 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847709 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:48:36.983928 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847712 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:48:36.983928 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847714 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:48:36.983928 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847719 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:48:36.983928 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847721 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:48:36.983928 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847724 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:48:36.983928 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847726 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:48:36.983928 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847729 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:48:36.983928 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847731 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:48:36.983928 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847734 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:48:36.983928 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847736 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:48:36.983928 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847739 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:48:36.983928 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847741 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:48:36.985012 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847744 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:48:36.985012 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847746 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:48:36.985012 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847749 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:48:36.985012 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847752 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:48:36.985012 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847754 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:48:36.985012 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847757 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:48:36.985012 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847760 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:48:36.985012 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847762 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:48:36.985012 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847765 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:48:36.985012 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847767 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:48:36.985012 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847769 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:48:36.985012 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847772 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:48:36.985012 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847776 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:48:36.985012 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847778 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:48:36.985012 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847781 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:48:36.985012 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847783 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:48:36.985012 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847786 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:48:36.985012 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847789 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:48:36.985012 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847791 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:48:36.985012 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.847794 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:48:36.986110 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.847799 2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 23:48:36.986110 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.892028 2553 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 23:48:36.986110 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.892046 2553 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 23:48:36.986110 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.892072 2553 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 23:48:36.986110 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.892079 2553 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 23:48:36.986110 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892158 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:48:36.986110 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892164 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:48:36.986110 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892168 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:48:36.986110 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892171 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:48:36.986110 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892174 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:48:36.986110 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892177 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:48:36.986110 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892180 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:48:36.986110 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892183 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:48:36.986110 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892185 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:48:36.986970 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892188 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:48:36.986970 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892191 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:48:36.986970 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892194 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:48:36.986970 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892197 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:48:36.986970 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892200 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:48:36.986970 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892203 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:48:36.986970 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892206 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:48:36.986970 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892209 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:48:36.986970 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892213 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:48:36.986970 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892215 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:48:36.986970 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892218 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:48:36.986970 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892221 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:48:36.986970 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892223 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:48:36.986970 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892226 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:48:36.986970 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892230 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:48:36.986970 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892234 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:48:36.986970 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892238 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:48:36.986970 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892241 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:48:36.986970 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892244 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:48:36.988065 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892246 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:48:36.988065 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892250 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:48:36.988065 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892253 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:48:36.988065 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892256 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:48:36.988065 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892259 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:48:36.988065 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892263 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:48:36.988065 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892265 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:48:36.988065 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892268 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:48:36.988065 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892271 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:48:36.988065 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892274 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:48:36.988065 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892276 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:48:36.988065 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892279 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:48:36.988065 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892282 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:48:36.988065 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892285 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:48:36.988065 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892288 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:48:36.988065 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892291 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:48:36.988065 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892293 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:48:36.988065 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892296 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:48:36.988065 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892298 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:48:36.988948 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892301 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:48:36.988948 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892303 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:48:36.988948 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892306 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:48:36.988948 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892309 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:48:36.988948 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892311 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:48:36.988948 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892314 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:48:36.988948 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892317 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:48:36.988948 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892319 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:48:36.988948 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892322 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:48:36.988948 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892324 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:48:36.988948 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892327 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:48:36.988948 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892330 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:48:36.988948 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892333 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:48:36.988948 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892335 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:48:36.988948 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892338 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:48:36.988948 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892342 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:48:36.988948 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892347 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:48:36.988948 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892352 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:48:36.988948 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892356 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:48:36.989967 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892360 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:48:36.989967 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892363 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:48:36.989967 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892366 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:48:36.989967 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892369 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:48:36.989967 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892372 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:48:36.989967 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892374 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:48:36.989967 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892377 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:48:36.989967 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892379 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:48:36.989967 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892382 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:48:36.989967 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892386 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:48:36.989967 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892388 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:48:36.989967 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892391 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:48:36.989967 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892393 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:48:36.989967 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892404 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:48:36.989967 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892407 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:48:36.989967 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892410 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:48:36.989967 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892413 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:48:36.989967 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892416 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:48:36.989967 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892418 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:48:36.989967 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892421 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:48:36.989967 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892423 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:48:36.990832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892426 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:48:36.990832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892428 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:48:36.990832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892431 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:48:36.990832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892434 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:48:36.990832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892436 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:48:36.990832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892439 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:48:36.990832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892442 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:48:36.990832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892444 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:48:36.990832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892447 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:48:36.990832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892450 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:48:36.990832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892453 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:48:36.990832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892456 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:48:36.990832 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.892461 2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 23:48:36.990832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892578 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:48:36.990832 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892584 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:48:36.991688 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892587 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:48:36.991688 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892590 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:48:36.991688 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892593 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:48:36.991688 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892595 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:48:36.991688 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892598 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:48:36.991688 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892603 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:48:36.991688 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892607 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:48:36.991688 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892610 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:48:36.991688 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892613 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:48:36.991688 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892615 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:48:36.991688 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892624 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:48:36.991688 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892627 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:48:36.991688 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892630 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:48:36.991688 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892633 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:48:36.991688 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892635 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:48:36.991688 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892638 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:48:36.991688 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892641 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:48:36.991688 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892643 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:48:36.991688 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892646 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:48:36.992547 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892648 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:48:36.992547 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892651 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:48:36.992547 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892654 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:48:36.992547 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892657 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:48:36.992547 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892659 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:48:36.992547 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892662 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:48:36.992547 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892665 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:48:36.992547 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892667 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:48:36.992547 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892670 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:48:36.992547 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892673 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:48:36.992547 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892675 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:48:36.992547 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892678 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:48:36.992547 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892681 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:48:36.992547 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892683 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:48:36.992547 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892686 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:48:36.992547 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892688 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:48:36.992547 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892691 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:48:36.992547 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892694 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:48:36.992547 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892696 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:48:36.992547 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892698 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:48:36.993362 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892701 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:48:36.993362 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892703 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:48:36.993362 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892706 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:48:36.993362 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892708 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:48:36.993362 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892717 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:48:36.993362 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892719 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:48:36.993362 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892722 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:48:36.993362 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892725 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:48:36.993362 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892727 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:48:36.993362 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892730 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:48:36.993362 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892732 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:48:36.993362 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892734 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:48:36.993362 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892737 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:48:36.993362 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892740 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:48:36.993362 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892742 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:48:36.993362 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892745 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:48:36.993362 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892747 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:48:36.993362 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892750 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:48:36.993362 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892753 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:48:36.993362 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892755 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:48:36.993996 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892758 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:48:36.993996 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892760 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:48:36.993996 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892763 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:48:36.993996 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892767 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:48:36.993996 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892770 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:48:36.993996 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892773 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:48:36.993996 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892775 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:48:36.993996 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892778 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:48:36.993996 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892782 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:48:36.993996 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892786 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:48:36.993996 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892788 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:48:36.993996 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892792 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:48:36.993996 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892795 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:48:36.993996 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892798 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:48:36.993996 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892801 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:48:36.993996 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892803 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:48:36.993996 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892806 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:48:36.993996 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892810 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:48:36.993996 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892813 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:48:36.994632 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892815 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:48:36.994632 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892818 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:48:36.994632 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892820 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:48:36.994632 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892823 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:48:36.994632 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892825 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:48:36.994632 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892828 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:48:36.994632 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892830 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:48:36.994632 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892833 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:48:36.994632 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892835 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:48:36.994632 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892837 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:48:36.994632 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892840 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:48:36.994632 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892843 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:48:36.994632 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892846 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:48:36.994632 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892848 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:48:36.994632 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892851 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:48:36.994632 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892853 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:48:36.994632 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892856 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:48:36.994632 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892858 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:48:36.994632 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:36.892861 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:48:36.995281 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.892866 2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 23:48:36.995281 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.893006 2553 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 23:48:36.995281 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.895016 2553 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 23:48:36.995281 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.896106 2553 server.go:1019] "Starting client certificate rotation" Jul 06 23:48:36.995281 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.896217 2553 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 23:48:36.995281 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.896257 2553 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 23:48:36.995281 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.898579 2553 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 23:48:36.995281 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.900465 2553 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 23:48:36.995281 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.909055 2553 log.go:25] "Validated CRI v1 runtime API" Jul 06 23:48:36.995281 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.913103 2553 log.go:25] "Validated CRI v1 image API" Jul 06 23:48:36.995281 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.914007 2553 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 23:48:36.995281 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.916136 2553 fs.go:135] Filesystem UUIDs: map[2e43a560-dfa4-427f-bce3-7d1d61606c63:/dev/nvme0n1p4 6328be73-cedb-4d03-a27f-278422a9b227:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 06 23:48:36.995705 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.916152 2553 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 23:48:36.995923 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.922015 2553 manager.go:216] Machine: {Timestamp:2026-07-06 23:48:36.920865379 +0000 UTC m=+0.312772560 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099678 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec226badebf3ea31005f1694f50cbebf SystemUUID:ec226bad-ebf3-ea31-005f-1694f50cbebf BootID:b2b1b3d3-91cd-4681-9544-d1182747f7a3 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ee:25:02:8c:8d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ee:25:02:8c:8d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0a:0a:01:13:8e:44 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 23:48:36.995923 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.922152 2553 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 23:48:36.995923 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.922285 2553 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 23:48:36.995923 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.924454 2553 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 23:48:36.995923 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.924479 2553 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-25.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 23:48:36.995923 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.925065 2553 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 23:48:36.995923 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.925082 2553 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 23:48:36.995923 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.925090 2553 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.925607 2553 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.926871 2553 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.928851 2553 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.931427 2553 state_mem.go:36] "Initialized new in-memory state store" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.931549 2553 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.931576 2553 kubelet.go:486] "Attempting to sync node with API server" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.931585 2553 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.931596 2553 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.931605 2553 kubelet.go:398] "Adding apiserver pod source" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.931614 2553 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.931950 2553 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.931965 2553 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.935829 2553 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.937461 2553 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.937488 2553 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.938405 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.938419 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.938425 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.938430 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.938439 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.938445 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.938452 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.938457 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.938463 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.938470 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.938482 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.938490 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.939053 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.939061 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.942357 2553 server.go:1268] "Started kubelet" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.942450 2553 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.942458 2553 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.942522 2553 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.944238 2553 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-25.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.944475 2553 server.go:310] "Adding debug handlers to kubelet server" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:36.944736 2553 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-25.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:36.944736 2553 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 23:48:36.996353 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:36.947474 2553 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 23:48:36.999689 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:36.948931 2553 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-25.ec2.internal.18bfd8390946210b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-25.ec2.internal,UID:ip-10-0-136-25.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-25.ec2.internal,},FirstTimestamp:2026-07-06 23:48:36.942332171 +0000 UTC m=+0.334239329,LastTimestamp:2026-07-06 23:48:36.942332171 +0000 UTC m=+0.334239329,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-25.ec2.internal,}" Jul 06 23:48:36.999689 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.953737 2553 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 23:48:36.999689 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.957519 2553 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 23:48:36.999689 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.957530 2553 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 23:48:36.999689 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:36.957628 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-25.ec2.internal\" not found" Jul 06 23:48:36.999689 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.957985 2553 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 23:48:36.999689 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.957988 2553 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 23:48:36.999689 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.958008 2553 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 23:48:36.999689 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.958136 2553 reconstruct.go:97] "Volume reconstruction finished" Jul 06 23:48:36.999689 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.958147 2553 reconciler.go:29] "Reconciler: start to sync state" Jul 06 23:48:36.999689 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.958316 2553 factory.go:55] Registering systemd factory Jul 06 23:48:36.999689 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.958329 2553 factory.go:223] Registration of the systemd container factory successfully Jul 06 23:48:36.999689 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.958525 2553 factory.go:153] Registering CRI-O factory Jul 06 23:48:36.999689 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.958540 2553 factory.go:223] Registration of the crio container factory successfully Jul 06 23:48:36.999689 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.958589 2553 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 23:48:36.999689 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.958611 2553 factory.go:103] Registering Raw factory Jul 06 23:48:36.999689 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.958624 2553 manager.go:1196] Started watching for new ooms in manager Jul 06 23:48:36.999689 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:36.959479 2553 manager.go:349] Starting recovery of all containers Jul 06 23:48:36.999689 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:36.960159 2553 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 23:48:36.999689 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:36.960842 2553 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-25.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.002856 2553 manager.go:354] Recovery completed Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.007413 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.009310 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-25.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.009343 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-25.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.009357 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-25.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.009924 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.009925 2553 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.009966 2553 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.010024 2553 state_mem.go:36] "Initialized new in-memory state store" Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.010538 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-25.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.010562 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-25.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.010571 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-25.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.012220 2553 policy_none.go:49] "None policy: Start" Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.012234 2553 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.012244 2553 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.012306 2553 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-67hzr" Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.014250 2553 policy_none.go:47] "Start" Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:37.022897 2553 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-25.ec2.internal.18bfd8390d44634b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-25.ec2.internal,UID:ip-10-0-136-25.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-25.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-25.ec2.internal,},FirstTimestamp:2026-07-06 23:48:37.009326923 +0000 UTC m=+0.401234085,LastTimestamp:2026-07-06 23:48:37.009326923 +0000 UTC m=+0.401234085,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-25.ec2.internal,}" Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.023037 2553 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-67hzr" Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:37.057741 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-25.ec2.internal\" not found" Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.072718 2553 manager.go:337] "Starting Device Plugin manager" Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:37.072756 2553 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.072766 2553 server.go:87] "Starting device plugin registration server" Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.073045 2553 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.073062 2553 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.073158 2553 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.073238 2553 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.073249 2553 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:37.073814 2553 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 23:48:37.095926 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:37.073854 2553 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-25.ec2.internal\" not found" Jul 06 23:48:37.123999 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.112824 2553 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 23:48:37.123999 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.113890 2553 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 23:48:37.123999 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.113933 2553 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 23:48:37.123999 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.113962 2553 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 23:48:37.123999 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:37.114004 2553 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 23:48:37.123999 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.116808 2553 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:48:37.243559 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:37.168132 2553 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-136-25.ec2.internal\" not found" node="ip-10-0-136-25.ec2.internal" Jul 06 23:48:37.243559 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.173144 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:48:37.243559 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.174045 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-25.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:48:37.243559 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.174073 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-25.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:48:37.243559 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.174085 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-25.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:48:37.243559 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.174107 2553 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-25.ec2.internal" Jul 06 23:48:37.243559 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.189173 2553 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-25.ec2.internal" Jul 06 23:48:37.243559 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:37.189196 2553 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-25.ec2.internal\": node \"ip-10-0-136-25.ec2.internal\" not found" Jul 06 23:48:37.243559 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.214292 2553 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-25.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-25.ec2.internal"] Jul 06 23:48:37.243559 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.214375 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:48:37.243559 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.215334 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-25.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:48:37.243559 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.215361 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-25.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:48:37.243559 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.215372 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-25.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:48:37.243559 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.217806 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:48:37.243559 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.217924 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-25.ec2.internal" Jul 06 23:48:37.243559 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.217953 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:48:37.243559 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.218507 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-25.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:48:37.243559 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.218535 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-25.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:48:37.243559 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.218546 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-25.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:48:37.243559 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.218509 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-25.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:48:37.243559 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.218607 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-25.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:48:37.243559 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.218618 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-25.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:48:37.243559 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.221547 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-25.ec2.internal" Jul 06 23:48:37.243559 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.221573 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:48:37.243559 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.222392 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-25.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:48:37.243559 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.222417 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-25.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:48:37.243559 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.222430 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-25.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:48:37.249566 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:37.248998 2553 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-25.ec2.internal\" not found" node="ip-10-0-136-25.ec2.internal" Jul 06 23:48:38.561862 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:37.253464 2553 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-25.ec2.internal\" not found" node="ip-10-0-136-25.ec2.internal" Jul 06 23:48:38.561862 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.260193 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ba668ff1926990fccf3f73e0625e305c-config\") pod \"kube-apiserver-proxy-ip-10-0-136-25.ec2.internal\" (UID: \"ba668ff1926990fccf3f73e0625e305c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-25.ec2.internal" Jul 06 23:48:38.561862 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.260219 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2a2f7f2a1d78cf5fb862a37fcc75004a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-25.ec2.internal\" (UID: \"2a2f7f2a1d78cf5fb862a37fcc75004a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-25.ec2.internal" Jul 06 23:48:38.561862 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:37.260237 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2a2f7f2a1d78cf5fb862a37fcc75004a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-25.ec2.internal\" (UID: \"2a2f7f2a1d78cf5fb862a37fcc75004a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-25.ec2.internal" Jul 06 23:48:38.561862 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:38.471164 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-25.ec2.internal\" not found" Jul 06 23:48:38.561862 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:38.543159 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ba668ff1926990fccf3f73e0625e305c-config\") pod \"kube-apiserver-proxy-ip-10-0-136-25.ec2.internal\" (UID: \"ba668ff1926990fccf3f73e0625e305c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-25.ec2.internal" Jul 06 23:48:38.561862 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:38.543209 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2a2f7f2a1d78cf5fb862a37fcc75004a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-25.ec2.internal\" (UID: \"2a2f7f2a1d78cf5fb862a37fcc75004a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-25.ec2.internal" Jul 06 23:48:38.561862 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:38.543225 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2a2f7f2a1d78cf5fb862a37fcc75004a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-25.ec2.internal\" (UID: \"2a2f7f2a1d78cf5fb862a37fcc75004a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-25.ec2.internal" Jul 06 23:48:38.561862 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:38.543267 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ba668ff1926990fccf3f73e0625e305c-config\") pod \"kube-apiserver-proxy-ip-10-0-136-25.ec2.internal\" (UID: \"ba668ff1926990fccf3f73e0625e305c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-25.ec2.internal" Jul 06 23:48:38.561862 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:38.543284 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2a2f7f2a1d78cf5fb862a37fcc75004a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-25.ec2.internal\" (UID: \"2a2f7f2a1d78cf5fb862a37fcc75004a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-25.ec2.internal" Jul 06 23:48:38.561862 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:38.543275 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2a2f7f2a1d78cf5fb862a37fcc75004a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-25.ec2.internal\" (UID: \"2a2f7f2a1d78cf5fb862a37fcc75004a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-25.ec2.internal" Jul 06 23:48:38.572538 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:38.571703 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-25.ec2.internal\" not found" Jul 06 23:48:38.744026 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:38.693534 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-25.ec2.internal\" not found" Jul 06 23:48:38.744026 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:38.733698 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-25.ec2.internal" Jul 06 23:48:38.849450 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:38.763307 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-25.ec2.internal" Jul 06 23:48:38.849450 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:38.794773 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-25.ec2.internal\" not found" Jul 06 23:48:38.980498 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:38.895246 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-25.ec2.internal\" not found" Jul 06 23:48:39.094920 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:38.995755 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-25.ec2.internal\" not found" Jul 06 23:48:39.094920 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:39.078269 2553 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 23:48:39.094920 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:39.078448 2553 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 23:48:39.096878 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:39.096686 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-25.ec2.internal\" not found" Jul 06 23:48:39.195246 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:39.113008 2553 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:48:39.198182 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:39.196901 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-25.ec2.internal\" not found" Jul 06 23:48:39.294899 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:39.212755 2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 23:43:38 +0000 UTC" deadline="2028-01-23 17:16:29.341190759 +0000 UTC" Jul 06 23:48:39.294899 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:39.212782 2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13577h27m50.128411673s" Jul 06 23:48:39.294899 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:39.212849 2553 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:48:39.383727 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:39.297900 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-25.ec2.internal\" not found" Jul 06 23:48:39.383727 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:39.308031 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:48:39.383727 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:39.309393 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-25.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:48:39.383727 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:39.309428 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-25.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:48:39.383727 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:39.309441 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-25.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:48:39.398166 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:39.398120 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-25.ec2.internal\" not found" Jul 06 23:48:39.416757 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:39.416723 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba668ff1926990fccf3f73e0625e305c.slice/crio-679b7d0d3ef62da41bc85aaa0c1357c106eefb3069985bf5a148b4eeaf55ab23 WatchSource:0}: Error finding container 679b7d0d3ef62da41bc85aaa0c1357c106eefb3069985bf5a148b4eeaf55ab23: Status 404 returned error can't find the container with id 679b7d0d3ef62da41bc85aaa0c1357c106eefb3069985bf5a148b4eeaf55ab23 Jul 06 23:48:39.417806 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:39.417779 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a2f7f2a1d78cf5fb862a37fcc75004a.slice/crio-8eee7b8da15133628ede0ab4364d9a7f0fd9a37bda055d8bddc2823027378858 WatchSource:0}: Error finding container 8eee7b8da15133628ede0ab4364d9a7f0fd9a37bda055d8bddc2823027378858: Status 404 returned error can't find the container with id 8eee7b8da15133628ede0ab4364d9a7f0fd9a37bda055d8bddc2823027378858 Jul 06 23:48:39.421280 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:39.421264 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 23:48:39.498646 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:39.498596 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-25.ec2.internal\" not found" Jul 06 23:48:39.599321 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:39.599228 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-25.ec2.internal\" not found" Jul 06 23:48:39.636049 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:39.636020 2553 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:48:39.640953 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:39.640925 2553 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-25.ec2.internal" Jul 06 23:48:39.656889 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:39.656860 2553 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 23:48:39.657682 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:39.657666 2553 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-25.ec2.internal" Jul 06 23:48:39.665053 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:39.665033 2553 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 23:48:40.114425 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.114373 2553 apiserver.go:52] "Watching apiserver" Jul 06 23:48:40.123438 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.123235 2553 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 23:48:40.123622 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.123590 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-zmkd8","openshift-network-operator/iptables-alerter-pwvps","kube-system/global-pull-secret-syncer-55rgj","kube-system/kube-apiserver-proxy-ip-10-0-136-25.ec2.internal","openshift-dns/node-resolver-9tms8","openshift-image-registry/node-ca-tdk4b","openshift-network-diagnostics/network-check-target-qndcn","openshift-ovn-kubernetes/ovnkube-node-4z8vq","kube-system/konnectivity-agent-j58wh","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2lwj","openshift-cluster-node-tuning-operator/tuned-8b784","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-25.ec2.internal","openshift-multus/multus-additional-cni-plugins-6r2vp","openshift-multus/multus-p274v"] Jul 06 23:48:40.126150 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.126126 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2lwj" Jul 06 23:48:40.128451 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.128428 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6r2vp" Jul 06 23:48:40.129144 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.129127 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 23:48:40.129231 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.129162 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 23:48:40.129231 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.129203 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 23:48:40.129231 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.129216 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dwzqw\"" Jul 06 23:48:40.131234 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.131214 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:48:40.131336 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:40.131286 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55rgj" podUID="68c477a5-72a1-47ec-9864-17cd0021a5f9" Jul 06 23:48:40.131401 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.131346 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 23:48:40.131401 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.131380 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vn7cp\"" Jul 06 23:48:40.131505 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.131406 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 23:48:40.131555 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.131536 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 23:48:40.131817 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.131660 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 23:48:40.131817 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.131728 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 23:48:40.133782 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.133736 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-p274v" Jul 06 23:48:40.136235 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.136194 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9tms8" Jul 06 23:48:40.136508 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.136392 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9bl6t\"" Jul 06 23:48:40.136508 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.136462 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 23:48:40.138531 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.138512 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tdk4b" Jul 06 23:48:40.138884 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.138863 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 23:48:40.139162 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.139142 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 23:48:40.139247 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.139169 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-29nl9\"" Jul 06 23:48:40.141259 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.141242 2553 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 23:48:40.141610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.141422 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zrwzl\"" Jul 06 23:48:40.141610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.141455 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 23:48:40.141610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.141457 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 23:48:40.141779 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.141246 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j58wh" Jul 06 23:48:40.141779 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.141741 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 23:48:40.143044 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.143024 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:48:40.144129 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:40.143540 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qndcn" podUID="8f45c29d-2b22-4a11-bdc7-2e8dc27aab32" Jul 06 23:48:40.144597 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.144229 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9ngsl\"" Jul 06 23:48:40.144597 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.144320 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 23:48:40.144597 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.144229 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 23:48:40.146358 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.146333 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pwvps" Jul 06 23:48:40.148755 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.148733 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 23:48:40.148849 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.148828 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:48:40.149210 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:40.148907 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zmkd8" podUID="971187a6-5709-48db-b990-0a13db9fad13" Jul 06 23:48:40.149210 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.149032 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 23:48:40.149210 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.149081 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 23:48:40.149554 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.149535 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wxpdt\"" Jul 06 23:48:40.151477 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.151459 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.153695 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.153674 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.155278 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.155033 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wdjr5\"" Jul 06 23:48:40.155278 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.155084 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 23:48:40.155579 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.155559 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 23:48:40.156394 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.156377 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 23:48:40.156611 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.156389 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 23:48:40.156922 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.156906 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 23:48:40.157903 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.157883 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 23:48:40.158009 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.157964 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 23:48:40.158149 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.158013 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 23:48:40.158495 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.158336 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9gc2j\"" Jul 06 23:48:40.158825 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.158645 2553 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 23:48:40.170117 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.169967 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/991cd81c-10fa-4837-8cb9-e727879aa882-registration-dir\") pod \"aws-ebs-csi-driver-node-t2lwj\" (UID: \"991cd81c-10fa-4837-8cb9-e727879aa882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2lwj" Jul 06 23:48:40.170117 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170007 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d56f5\" (UniqueName: \"kubernetes.io/projected/4f2c8799-a48e-4f4c-bce4-7f0187c24298-kube-api-access-d56f5\") pod \"node-ca-tdk4b\" (UID: \"4f2c8799-a48e-4f4c-bce4-7f0187c24298\") " pod="openshift-image-registry/node-ca-tdk4b" Jul 06 23:48:40.170117 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170034 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/43908b50-c882-4e43-8d91-107f2acaf7b5-system-cni-dir\") pod \"multus-additional-cni-plugins-6r2vp\" (UID: \"43908b50-c882-4e43-8d91-107f2acaf7b5\") " pod="openshift-multus/multus-additional-cni-plugins-6r2vp" Jul 06 23:48:40.170117 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170058 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/43908b50-c882-4e43-8d91-107f2acaf7b5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6r2vp\" (UID: \"43908b50-c882-4e43-8d91-107f2acaf7b5\") " pod="openshift-multus/multus-additional-cni-plugins-6r2vp" Jul 06 23:48:40.170117 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170083 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-host-var-lib-kubelet\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.170117 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170107 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/43908b50-c882-4e43-8d91-107f2acaf7b5-os-release\") pod \"multus-additional-cni-plugins-6r2vp\" (UID: \"43908b50-c882-4e43-8d91-107f2acaf7b5\") " pod="openshift-multus/multus-additional-cni-plugins-6r2vp" Jul 06 23:48:40.170485 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170131 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2r4w\" (UniqueName: \"kubernetes.io/projected/43908b50-c882-4e43-8d91-107f2acaf7b5-kube-api-access-w2r4w\") pod \"multus-additional-cni-plugins-6r2vp\" (UID: \"43908b50-c882-4e43-8d91-107f2acaf7b5\") " pod="openshift-multus/multus-additional-cni-plugins-6r2vp" Jul 06 23:48:40.170485 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170156 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-multus-socket-dir-parent\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.170485 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170181 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-host-run-netns\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.170485 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170203 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-hostroot\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.170485 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170225 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-multus-conf-dir\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.170485 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170266 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-multus-daemon-config\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.170485 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170343 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6ntz\" (UniqueName: \"kubernetes.io/projected/93308370-a961-4b85-a764-abac1d17c520-kube-api-access-n6ntz\") pod \"node-resolver-9tms8\" (UID: \"93308370-a961-4b85-a764-abac1d17c520\") " pod="openshift-dns/node-resolver-9tms8" Jul 06 23:48:40.170485 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170388 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/991cd81c-10fa-4837-8cb9-e727879aa882-etc-selinux\") pod \"aws-ebs-csi-driver-node-t2lwj\" (UID: \"991cd81c-10fa-4837-8cb9-e727879aa882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2lwj" Jul 06 23:48:40.170485 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170412 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/991cd81c-10fa-4837-8cb9-e727879aa882-sys-fs\") pod \"aws-ebs-csi-driver-node-t2lwj\" (UID: \"991cd81c-10fa-4837-8cb9-e727879aa882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2lwj" Jul 06 23:48:40.170485 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170450 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/43908b50-c882-4e43-8d91-107f2acaf7b5-cnibin\") pod \"multus-additional-cni-plugins-6r2vp\" (UID: \"43908b50-c882-4e43-8d91-107f2acaf7b5\") " pod="openshift-multus/multus-additional-cni-plugins-6r2vp" Jul 06 23:48:40.170485 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170478 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/43908b50-c882-4e43-8d91-107f2acaf7b5-cni-binary-copy\") pod \"multus-additional-cni-plugins-6r2vp\" (UID: \"43908b50-c882-4e43-8d91-107f2acaf7b5\") " pod="openshift-multus/multus-additional-cni-plugins-6r2vp" Jul 06 23:48:40.171838 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170502 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/43908b50-c882-4e43-8d91-107f2acaf7b5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6r2vp\" (UID: \"43908b50-c882-4e43-8d91-107f2acaf7b5\") " pod="openshift-multus/multus-additional-cni-plugins-6r2vp" Jul 06 23:48:40.171838 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170526 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff6ba6b4-93ca-481f-8193-dfedf07c3875-agent-certs\") pod \"konnectivity-agent-j58wh\" (UID: \"ff6ba6b4-93ca-481f-8193-dfedf07c3875\") " pod="kube-system/konnectivity-agent-j58wh" Jul 06 23:48:40.171838 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170570 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-multus-cni-dir\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.171838 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170595 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-os-release\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.171838 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170621 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/991cd81c-10fa-4837-8cb9-e727879aa882-socket-dir\") pod \"aws-ebs-csi-driver-node-t2lwj\" (UID: \"991cd81c-10fa-4837-8cb9-e727879aa882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2lwj" Jul 06 23:48:40.171838 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170646 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-system-cni-dir\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.171838 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170669 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-cni-binary-copy\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.171838 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170692 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-host-run-k8s-cni-cncf-io\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.171838 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170717 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-host-var-lib-cni-multus\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.171838 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170741 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-host-run-multus-certs\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.171838 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170816 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-etc-kubernetes\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.171838 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170841 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/93308370-a961-4b85-a764-abac1d17c520-hosts-file\") pod \"node-resolver-9tms8\" (UID: \"93308370-a961-4b85-a764-abac1d17c520\") " pod="openshift-dns/node-resolver-9tms8" Jul 06 23:48:40.171838 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170863 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/991cd81c-10fa-4837-8cb9-e727879aa882-device-dir\") pod \"aws-ebs-csi-driver-node-t2lwj\" (UID: \"991cd81c-10fa-4837-8cb9-e727879aa882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2lwj" Jul 06 23:48:40.171838 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170890 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmfjh\" (UniqueName: \"kubernetes.io/projected/991cd81c-10fa-4837-8cb9-e727879aa882-kube-api-access-gmfjh\") pod \"aws-ebs-csi-driver-node-t2lwj\" (UID: \"991cd81c-10fa-4837-8cb9-e727879aa882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2lwj" Jul 06 23:48:40.174736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170916 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/68c477a5-72a1-47ec-9864-17cd0021a5f9-kubelet-config\") pod \"global-pull-secret-syncer-55rgj\" (UID: \"68c477a5-72a1-47ec-9864-17cd0021a5f9\") " pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:48:40.174736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170950 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68c477a5-72a1-47ec-9864-17cd0021a5f9-original-pull-secret\") pod \"global-pull-secret-syncer-55rgj\" (UID: \"68c477a5-72a1-47ec-9864-17cd0021a5f9\") " pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:48:40.174736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.170977 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93308370-a961-4b85-a764-abac1d17c520-tmp-dir\") pod \"node-resolver-9tms8\" (UID: \"93308370-a961-4b85-a764-abac1d17c520\") " pod="openshift-dns/node-resolver-9tms8" Jul 06 23:48:40.174736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.171003 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4f2c8799-a48e-4f4c-bce4-7f0187c24298-host\") pod \"node-ca-tdk4b\" (UID: \"4f2c8799-a48e-4f4c-bce4-7f0187c24298\") " pod="openshift-image-registry/node-ca-tdk4b" Jul 06 23:48:40.174736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.171029 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4f2c8799-a48e-4f4c-bce4-7f0187c24298-serviceca\") pod \"node-ca-tdk4b\" (UID: \"4f2c8799-a48e-4f4c-bce4-7f0187c24298\") " pod="openshift-image-registry/node-ca-tdk4b" Jul 06 23:48:40.174736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.171054 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff6ba6b4-93ca-481f-8193-dfedf07c3875-konnectivity-ca\") pod \"konnectivity-agent-j58wh\" (UID: \"ff6ba6b4-93ca-481f-8193-dfedf07c3875\") " pod="kube-system/konnectivity-agent-j58wh" Jul 06 23:48:40.174736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.171104 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-host-var-lib-cni-bin\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.174736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.171141 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4567\" (UniqueName: \"kubernetes.io/projected/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-kube-api-access-p4567\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.174736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.171192 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/991cd81c-10fa-4837-8cb9-e727879aa882-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t2lwj\" (UID: \"991cd81c-10fa-4837-8cb9-e727879aa882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2lwj" Jul 06 23:48:40.174736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.171228 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/43908b50-c882-4e43-8d91-107f2acaf7b5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6r2vp\" (UID: \"43908b50-c882-4e43-8d91-107f2acaf7b5\") " pod="openshift-multus/multus-additional-cni-plugins-6r2vp" Jul 06 23:48:40.174736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.171253 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/68c477a5-72a1-47ec-9864-17cd0021a5f9-dbus\") pod \"global-pull-secret-syncer-55rgj\" (UID: \"68c477a5-72a1-47ec-9864-17cd0021a5f9\") " pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:48:40.174736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.171285 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sp29w\" (UniqueName: \"kubernetes.io/projected/8f45c29d-2b22-4a11-bdc7-2e8dc27aab32-kube-api-access-sp29w\") pod \"network-check-target-qndcn\" (UID: \"8f45c29d-2b22-4a11-bdc7-2e8dc27aab32\") " pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:48:40.174736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.171325 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-cnibin\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.236091 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.236053 2553 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ksnvp" Jul 06 23:48:40.241498 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.241465 2553 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 23:48:40.245927 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.245892 2553 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ksnvp" Jul 06 23:48:40.272031 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272004 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-host-var-lib-kubelet\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.272218 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272040 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4f2c8799-a48e-4f4c-bce4-7f0187c24298-serviceca\") pod \"node-ca-tdk4b\" (UID: \"4f2c8799-a48e-4f4c-bce4-7f0187c24298\") " pod="openshift-image-registry/node-ca-tdk4b" Jul 06 23:48:40.272218 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272062 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-etc-openvswitch\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.272218 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272083 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/43908b50-c882-4e43-8d91-107f2acaf7b5-os-release\") pod \"multus-additional-cni-plugins-6r2vp\" (UID: \"43908b50-c882-4e43-8d91-107f2acaf7b5\") " pod="openshift-multus/multus-additional-cni-plugins-6r2vp" Jul 06 23:48:40.272218 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272107 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2r4w\" (UniqueName: \"kubernetes.io/projected/43908b50-c882-4e43-8d91-107f2acaf7b5-kube-api-access-w2r4w\") pod \"multus-additional-cni-plugins-6r2vp\" (UID: \"43908b50-c882-4e43-8d91-107f2acaf7b5\") " pod="openshift-multus/multus-additional-cni-plugins-6r2vp" Jul 06 23:48:40.272218 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272130 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-multus-socket-dir-parent\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.272218 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272139 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-host-var-lib-kubelet\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.272218 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272151 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-host-run-netns\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.272218 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272194 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-host-run-netns\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.272218 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272209 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/43908b50-c882-4e43-8d91-107f2acaf7b5-os-release\") pod \"multus-additional-cni-plugins-6r2vp\" (UID: \"43908b50-c882-4e43-8d91-107f2acaf7b5\") " pod="openshift-multus/multus-additional-cni-plugins-6r2vp" Jul 06 23:48:40.272835 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272212 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-hostroot\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.272835 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272249 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-hostroot\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.272835 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272266 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-multus-conf-dir\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.272835 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272308 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/43908b50-c882-4e43-8d91-107f2acaf7b5-cnibin\") pod \"multus-additional-cni-plugins-6r2vp\" (UID: \"43908b50-c882-4e43-8d91-107f2acaf7b5\") " pod="openshift-multus/multus-additional-cni-plugins-6r2vp" Jul 06 23:48:40.272835 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272331 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-multus-socket-dir-parent\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.272835 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272357 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-multus-conf-dir\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.272835 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272338 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/43908b50-c882-4e43-8d91-107f2acaf7b5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6r2vp\" (UID: \"43908b50-c882-4e43-8d91-107f2acaf7b5\") " pod="openshift-multus/multus-additional-cni-plugins-6r2vp" Jul 06 23:48:40.272835 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272397 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-multus-cni-dir\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.272835 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272401 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/43908b50-c882-4e43-8d91-107f2acaf7b5-cnibin\") pod \"multus-additional-cni-plugins-6r2vp\" (UID: \"43908b50-c882-4e43-8d91-107f2acaf7b5\") " pod="openshift-multus/multus-additional-cni-plugins-6r2vp" Jul 06 23:48:40.272835 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272426 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-run-ovn\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.272835 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272452 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-multus-daemon-config\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.272835 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272477 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6ntz\" (UniqueName: \"kubernetes.io/projected/93308370-a961-4b85-a764-abac1d17c520-kube-api-access-n6ntz\") pod \"node-resolver-9tms8\" (UID: \"93308370-a961-4b85-a764-abac1d17c520\") " pod="openshift-dns/node-resolver-9tms8" Jul 06 23:48:40.272835 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272494 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/43908b50-c882-4e43-8d91-107f2acaf7b5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6r2vp\" (UID: \"43908b50-c882-4e43-8d91-107f2acaf7b5\") " pod="openshift-multus/multus-additional-cni-plugins-6r2vp" Jul 06 23:48:40.272835 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272501 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-host-cni-bin\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.272835 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272534 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/971187a6-5709-48db-b990-0a13db9fad13-metrics-certs\") pod \"network-metrics-daemon-zmkd8\" (UID: \"971187a6-5709-48db-b990-0a13db9fad13\") " pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:48:40.273658 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272552 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-multus-cni-dir\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.273658 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272556 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/991cd81c-10fa-4837-8cb9-e727879aa882-socket-dir\") pod \"aws-ebs-csi-driver-node-t2lwj\" (UID: \"991cd81c-10fa-4837-8cb9-e727879aa882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2lwj" Jul 06 23:48:40.273658 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272593 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4f2c8799-a48e-4f4c-bce4-7f0187c24298-serviceca\") pod \"node-ca-tdk4b\" (UID: \"4f2c8799-a48e-4f4c-bce4-7f0187c24298\") " pod="openshift-image-registry/node-ca-tdk4b" Jul 06 23:48:40.273658 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272598 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-cni-binary-copy\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.273658 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272627 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-host-var-lib-cni-multus\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.273658 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272638 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/991cd81c-10fa-4837-8cb9-e727879aa882-socket-dir\") pod \"aws-ebs-csi-driver-node-t2lwj\" (UID: \"991cd81c-10fa-4837-8cb9-e727879aa882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2lwj" Jul 06 23:48:40.273658 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272649 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4f2c8799-a48e-4f4c-bce4-7f0187c24298-host\") pod \"node-ca-tdk4b\" (UID: \"4f2c8799-a48e-4f4c-bce4-7f0187c24298\") " pod="openshift-image-registry/node-ca-tdk4b" Jul 06 23:48:40.273658 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272674 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-host-cni-netd\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.273658 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272678 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-host-var-lib-cni-multus\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.273658 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272701 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4f2c8799-a48e-4f4c-bce4-7f0187c24298-host\") pod \"node-ca-tdk4b\" (UID: \"4f2c8799-a48e-4f4c-bce4-7f0187c24298\") " pod="openshift-image-registry/node-ca-tdk4b" Jul 06 23:48:40.273658 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272701 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.273658 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272743 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rdmc\" (UniqueName: \"kubernetes.io/projected/3253bdfe-5248-4d4c-9864-11660eb042ca-kube-api-access-5rdmc\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.273658 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272771 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-etc-kubernetes\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.273658 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272802 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmfjh\" (UniqueName: \"kubernetes.io/projected/991cd81c-10fa-4837-8cb9-e727879aa882-kube-api-access-gmfjh\") pod \"aws-ebs-csi-driver-node-t2lwj\" (UID: \"991cd81c-10fa-4837-8cb9-e727879aa882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2lwj" Jul 06 23:48:40.274277 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272827 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/68c477a5-72a1-47ec-9864-17cd0021a5f9-kubelet-config\") pod \"global-pull-secret-syncer-55rgj\" (UID: \"68c477a5-72a1-47ec-9864-17cd0021a5f9\") " pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:48:40.274277 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272852 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68c477a5-72a1-47ec-9864-17cd0021a5f9-original-pull-secret\") pod \"global-pull-secret-syncer-55rgj\" (UID: \"68c477a5-72a1-47ec-9864-17cd0021a5f9\") " pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:48:40.274277 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272876 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93308370-a961-4b85-a764-abac1d17c520-tmp-dir\") pod \"node-resolver-9tms8\" (UID: \"93308370-a961-4b85-a764-abac1d17c520\") " pod="openshift-dns/node-resolver-9tms8" Jul 06 23:48:40.274277 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272881 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/68c477a5-72a1-47ec-9864-17cd0021a5f9-kubelet-config\") pod \"global-pull-secret-syncer-55rgj\" (UID: \"68c477a5-72a1-47ec-9864-17cd0021a5f9\") " pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:48:40.274277 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272897 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-host-slash\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.274277 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272920 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-host-run-ovn-kubernetes\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.274277 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272943 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/056e740d-60e4-41f3-9f77-d5b292d6ef59-iptables-alerter-script\") pod \"iptables-alerter-pwvps\" (UID: \"056e740d-60e4-41f3-9f77-d5b292d6ef59\") " pod="openshift-network-operator/iptables-alerter-pwvps" Jul 06 23:48:40.274277 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:40.272960 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:40.274277 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272968 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-etc-tuned\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.274277 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.272992 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4567\" (UniqueName: \"kubernetes.io/projected/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-kube-api-access-p4567\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.274277 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:40.273008 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68c477a5-72a1-47ec-9864-17cd0021a5f9-original-pull-secret podName:68c477a5-72a1-47ec-9864-17cd0021a5f9 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:40.772993351 +0000 UTC m=+2.982722853 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/68c477a5-72a1-47ec-9864-17cd0021a5f9-original-pull-secret") pod "global-pull-secret-syncer-55rgj" (UID: "68c477a5-72a1-47ec-9864-17cd0021a5f9") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:40.274277 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.273027 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-var-lib-openvswitch\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.274277 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.273043 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3253bdfe-5248-4d4c-9864-11660eb042ca-ovnkube-config\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.274277 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.273058 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-etc-sysctl-d\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.274986 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.273073 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-etc-sysctl-conf\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.274986 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.273081 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-multus-daemon-config\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.274986 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.273089 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-host\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.274986 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.273190 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/68c477a5-72a1-47ec-9864-17cd0021a5f9-dbus\") pod \"global-pull-secret-syncer-55rgj\" (UID: \"68c477a5-72a1-47ec-9864-17cd0021a5f9\") " pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:48:40.274986 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.273237 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp29w\" (UniqueName: \"kubernetes.io/projected/8f45c29d-2b22-4a11-bdc7-2e8dc27aab32-kube-api-access-sp29w\") pod \"network-check-target-qndcn\" (UID: \"8f45c29d-2b22-4a11-bdc7-2e8dc27aab32\") " pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:48:40.274986 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.273257 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/68c477a5-72a1-47ec-9864-17cd0021a5f9-dbus\") pod \"global-pull-secret-syncer-55rgj\" (UID: \"68c477a5-72a1-47ec-9864-17cd0021a5f9\") " pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:48:40.274986 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.273281 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-cnibin\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.274986 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.273387 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93308370-a961-4b85-a764-abac1d17c520-tmp-dir\") pod \"node-resolver-9tms8\" (UID: \"93308370-a961-4b85-a764-abac1d17c520\") " pod="openshift-dns/node-resolver-9tms8" Jul 06 23:48:40.274986 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.273432 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-cnibin\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.274986 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.273399 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-systemd-units\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.274986 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.273828 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3253bdfe-5248-4d4c-9864-11660eb042ca-ovn-node-metrics-cert\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.274986 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.273848 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-run\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.274986 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.273868 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/991cd81c-10fa-4837-8cb9-e727879aa882-registration-dir\") pod \"aws-ebs-csi-driver-node-t2lwj\" (UID: \"991cd81c-10fa-4837-8cb9-e727879aa882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2lwj" Jul 06 23:48:40.274986 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.273873 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-cni-binary-copy\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.274986 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.273886 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d56f5\" (UniqueName: \"kubernetes.io/projected/4f2c8799-a48e-4f4c-bce4-7f0187c24298-kube-api-access-d56f5\") pod \"node-ca-tdk4b\" (UID: \"4f2c8799-a48e-4f4c-bce4-7f0187c24298\") " pod="openshift-image-registry/node-ca-tdk4b" Jul 06 23:48:40.275683 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.273909 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3253bdfe-5248-4d4c-9864-11660eb042ca-env-overrides\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.275683 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.273940 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/991cd81c-10fa-4837-8cb9-e727879aa882-registration-dir\") pod \"aws-ebs-csi-driver-node-t2lwj\" (UID: \"991cd81c-10fa-4837-8cb9-e727879aa882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2lwj" Jul 06 23:48:40.275683 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.273940 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/43908b50-c882-4e43-8d91-107f2acaf7b5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6r2vp\" (UID: \"43908b50-c882-4e43-8d91-107f2acaf7b5\") " pod="openshift-multus/multus-additional-cni-plugins-6r2vp" Jul 06 23:48:40.275683 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.273981 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-node-log\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.275683 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.274002 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-lib-modules\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.275683 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.274034 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-var-lib-kubelet\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.275683 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.274080 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/056e740d-60e4-41f3-9f77-d5b292d6ef59-host-slash\") pod \"iptables-alerter-pwvps\" (UID: \"056e740d-60e4-41f3-9f77-d5b292d6ef59\") " pod="openshift-network-operator/iptables-alerter-pwvps" Jul 06 23:48:40.275683 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.274107 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/991cd81c-10fa-4837-8cb9-e727879aa882-etc-selinux\") pod \"aws-ebs-csi-driver-node-t2lwj\" (UID: \"991cd81c-10fa-4837-8cb9-e727879aa882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2lwj" Jul 06 23:48:40.275683 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.274131 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/991cd81c-10fa-4837-8cb9-e727879aa882-sys-fs\") pod \"aws-ebs-csi-driver-node-t2lwj\" (UID: \"991cd81c-10fa-4837-8cb9-e727879aa882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2lwj" Jul 06 23:48:40.275683 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.274156 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/43908b50-c882-4e43-8d91-107f2acaf7b5-cni-binary-copy\") pod \"multus-additional-cni-plugins-6r2vp\" (UID: \"43908b50-c882-4e43-8d91-107f2acaf7b5\") " pod="openshift-multus/multus-additional-cni-plugins-6r2vp" Jul 06 23:48:40.275683 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.274180 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff6ba6b4-93ca-481f-8193-dfedf07c3875-agent-certs\") pod \"konnectivity-agent-j58wh\" (UID: \"ff6ba6b4-93ca-481f-8193-dfedf07c3875\") " pod="kube-system/konnectivity-agent-j58wh" Jul 06 23:48:40.275683 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.274204 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-os-release\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.275683 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.274227 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-host-run-k8s-cni-cncf-io\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.275683 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.274226 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/991cd81c-10fa-4837-8cb9-e727879aa882-etc-selinux\") pod \"aws-ebs-csi-driver-node-t2lwj\" (UID: \"991cd81c-10fa-4837-8cb9-e727879aa882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2lwj" Jul 06 23:48:40.276315 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.274224 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/991cd81c-10fa-4837-8cb9-e727879aa882-sys-fs\") pod \"aws-ebs-csi-driver-node-t2lwj\" (UID: \"991cd81c-10fa-4837-8cb9-e727879aa882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2lwj" Jul 06 23:48:40.276315 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.274266 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-etc-kubernetes\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.276315 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.274280 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-host-run-k8s-cni-cncf-io\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.276315 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.274309 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/93308370-a961-4b85-a764-abac1d17c520-hosts-file\") pod \"node-resolver-9tms8\" (UID: \"93308370-a961-4b85-a764-abac1d17c520\") " pod="openshift-dns/node-resolver-9tms8" Jul 06 23:48:40.276315 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.274335 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-system-cni-dir\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.276315 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.274360 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-host-run-multus-certs\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.276315 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.274386 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-host-kubelet\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.276315 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.274411 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-run-systemd\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.276315 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.274411 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-os-release\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.276315 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.274457 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-system-cni-dir\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.276315 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.274464 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-host-run-multus-certs\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.276315 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.274491 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3253bdfe-5248-4d4c-9864-11660eb042ca-ovnkube-script-lib\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.276315 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.274802 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-822d4\" (UniqueName: \"kubernetes.io/projected/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-kube-api-access-822d4\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.276315 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.274849 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/991cd81c-10fa-4837-8cb9-e727879aa882-device-dir\") pod \"aws-ebs-csi-driver-node-t2lwj\" (UID: \"991cd81c-10fa-4837-8cb9-e727879aa882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2lwj" Jul 06 23:48:40.276315 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.275166 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-etc-kubernetes\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.276955 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.275217 2553 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 23:48:40.276955 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.275320 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/43908b50-c882-4e43-8d91-107f2acaf7b5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6r2vp\" (UID: \"43908b50-c882-4e43-8d91-107f2acaf7b5\") " pod="openshift-multus/multus-additional-cni-plugins-6r2vp" Jul 06 23:48:40.276955 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.275371 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-host-run-netns\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.276955 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.275393 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/43908b50-c882-4e43-8d91-107f2acaf7b5-cni-binary-copy\") pod \"multus-additional-cni-plugins-6r2vp\" (UID: \"43908b50-c882-4e43-8d91-107f2acaf7b5\") " pod="openshift-multus/multus-additional-cni-plugins-6r2vp" Jul 06 23:48:40.276955 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.275405 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdgbg\" (UniqueName: \"kubernetes.io/projected/971187a6-5709-48db-b990-0a13db9fad13-kube-api-access-pdgbg\") pod \"network-metrics-daemon-zmkd8\" (UID: \"971187a6-5709-48db-b990-0a13db9fad13\") " pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:48:40.276955 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.275433 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-sys\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.276955 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.275463 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff6ba6b4-93ca-481f-8193-dfedf07c3875-konnectivity-ca\") pod \"konnectivity-agent-j58wh\" (UID: \"ff6ba6b4-93ca-481f-8193-dfedf07c3875\") " pod="kube-system/konnectivity-agent-j58wh" Jul 06 23:48:40.276955 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.275496 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-host-var-lib-cni-bin\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.276955 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.275501 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/93308370-a961-4b85-a764-abac1d17c520-hosts-file\") pod \"node-resolver-9tms8\" (UID: \"93308370-a961-4b85-a764-abac1d17c520\") " pod="openshift-dns/node-resolver-9tms8" Jul 06 23:48:40.276955 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.275521 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-etc-sysconfig\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.276955 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.275562 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-tmp\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.276955 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.275946 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff6ba6b4-93ca-481f-8193-dfedf07c3875-konnectivity-ca\") pod \"konnectivity-agent-j58wh\" (UID: \"ff6ba6b4-93ca-481f-8193-dfedf07c3875\") " pod="kube-system/konnectivity-agent-j58wh" Jul 06 23:48:40.276955 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.275999 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-host-var-lib-cni-bin\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.276955 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.276050 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/991cd81c-10fa-4837-8cb9-e727879aa882-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t2lwj\" (UID: \"991cd81c-10fa-4837-8cb9-e727879aa882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2lwj" Jul 06 23:48:40.277575 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.276092 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/43908b50-c882-4e43-8d91-107f2acaf7b5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6r2vp\" (UID: \"43908b50-c882-4e43-8d91-107f2acaf7b5\") " pod="openshift-multus/multus-additional-cni-plugins-6r2vp" Jul 06 23:48:40.277575 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.276127 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h584q\" (UniqueName: \"kubernetes.io/projected/056e740d-60e4-41f3-9f77-d5b292d6ef59-kube-api-access-h584q\") pod \"iptables-alerter-pwvps\" (UID: \"056e740d-60e4-41f3-9f77-d5b292d6ef59\") " pod="openshift-network-operator/iptables-alerter-pwvps" Jul 06 23:48:40.277575 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.276192 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-run-openvswitch\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.277575 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.276223 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-log-socket\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.277575 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.276253 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-etc-modprobe-d\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.277575 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.276404 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-etc-systemd\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.277575 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.276597 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/43908b50-c882-4e43-8d91-107f2acaf7b5-system-cni-dir\") pod \"multus-additional-cni-plugins-6r2vp\" (UID: \"43908b50-c882-4e43-8d91-107f2acaf7b5\") " pod="openshift-multus/multus-additional-cni-plugins-6r2vp" Jul 06 23:48:40.277575 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.276717 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/43908b50-c882-4e43-8d91-107f2acaf7b5-system-cni-dir\") pod \"multus-additional-cni-plugins-6r2vp\" (UID: \"43908b50-c882-4e43-8d91-107f2acaf7b5\") " pod="openshift-multus/multus-additional-cni-plugins-6r2vp" Jul 06 23:48:40.277575 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.276777 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/991cd81c-10fa-4837-8cb9-e727879aa882-kubelet-dir\") pod \"aws-ebs-csi-driver-node-t2lwj\" (UID: \"991cd81c-10fa-4837-8cb9-e727879aa882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2lwj" Jul 06 23:48:40.277575 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.277234 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/43908b50-c882-4e43-8d91-107f2acaf7b5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6r2vp\" (UID: \"43908b50-c882-4e43-8d91-107f2acaf7b5\") " pod="openshift-multus/multus-additional-cni-plugins-6r2vp" Jul 06 23:48:40.277575 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.277360 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/991cd81c-10fa-4837-8cb9-e727879aa882-device-dir\") pod \"aws-ebs-csi-driver-node-t2lwj\" (UID: \"991cd81c-10fa-4837-8cb9-e727879aa882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2lwj" Jul 06 23:48:40.279349 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.279284 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff6ba6b4-93ca-481f-8193-dfedf07c3875-agent-certs\") pod \"konnectivity-agent-j58wh\" (UID: \"ff6ba6b4-93ca-481f-8193-dfedf07c3875\") " pod="kube-system/konnectivity-agent-j58wh" Jul 06 23:48:40.282039 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:40.282018 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:48:40.282126 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:40.282049 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:48:40.282126 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:40.282063 2553 projected.go:196] Error preparing data for projected volume kube-api-access-sp29w for pod openshift-network-diagnostics/network-check-target-qndcn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:40.282126 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:40.282120 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f45c29d-2b22-4a11-bdc7-2e8dc27aab32-kube-api-access-sp29w podName:8f45c29d-2b22-4a11-bdc7-2e8dc27aab32 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:40.782102203 +0000 UTC m=+2.991831702 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-sp29w" (UniqueName: "kubernetes.io/projected/8f45c29d-2b22-4a11-bdc7-2e8dc27aab32-kube-api-access-sp29w") pod "network-check-target-qndcn" (UID: "8f45c29d-2b22-4a11-bdc7-2e8dc27aab32") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:40.284586 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.284565 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmfjh\" (UniqueName: \"kubernetes.io/projected/991cd81c-10fa-4837-8cb9-e727879aa882-kube-api-access-gmfjh\") pod \"aws-ebs-csi-driver-node-t2lwj\" (UID: \"991cd81c-10fa-4837-8cb9-e727879aa882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2lwj" Jul 06 23:48:40.284956 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.284934 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2r4w\" (UniqueName: \"kubernetes.io/projected/43908b50-c882-4e43-8d91-107f2acaf7b5-kube-api-access-w2r4w\") pod \"multus-additional-cni-plugins-6r2vp\" (UID: \"43908b50-c882-4e43-8d91-107f2acaf7b5\") " pod="openshift-multus/multus-additional-cni-plugins-6r2vp" Jul 06 23:48:40.285036 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.285003 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6ntz\" (UniqueName: \"kubernetes.io/projected/93308370-a961-4b85-a764-abac1d17c520-kube-api-access-n6ntz\") pod \"node-resolver-9tms8\" (UID: \"93308370-a961-4b85-a764-abac1d17c520\") " pod="openshift-dns/node-resolver-9tms8" Jul 06 23:48:40.285100 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.285085 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4567\" (UniqueName: \"kubernetes.io/projected/ff9e3a5f-ea1a-4715-87b1-2766a82bef28-kube-api-access-p4567\") pod \"multus-p274v\" (UID: \"ff9e3a5f-ea1a-4715-87b1-2766a82bef28\") " pod="openshift-multus/multus-p274v" Jul 06 23:48:40.285159 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.285106 2553 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:48:40.286490 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.286471 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d56f5\" (UniqueName: \"kubernetes.io/projected/4f2c8799-a48e-4f4c-bce4-7f0187c24298-kube-api-access-d56f5\") pod \"node-ca-tdk4b\" (UID: \"4f2c8799-a48e-4f4c-bce4-7f0187c24298\") " pod="openshift-image-registry/node-ca-tdk4b" Jul 06 23:48:40.301214 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.301159 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-25.ec2.internal" event={"ID":"2a2f7f2a1d78cf5fb862a37fcc75004a","Type":"ContainerStarted","Data":"8eee7b8da15133628ede0ab4364d9a7f0fd9a37bda055d8bddc2823027378858"} Jul 06 23:48:40.302559 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.302527 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-25.ec2.internal" event={"ID":"ba668ff1926990fccf3f73e0625e305c","Type":"ContainerStarted","Data":"679b7d0d3ef62da41bc85aaa0c1357c106eefb3069985bf5a148b4eeaf55ab23"} Jul 06 23:48:40.377276 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377189 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-etc-openvswitch\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.377276 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377235 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-run-ovn\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.377276 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377262 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-host-cni-bin\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.377533 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377285 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/971187a6-5709-48db-b990-0a13db9fad13-metrics-certs\") pod \"network-metrics-daemon-zmkd8\" (UID: \"971187a6-5709-48db-b990-0a13db9fad13\") " pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:48:40.377533 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377327 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-host-cni-netd\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.377533 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377329 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-etc-openvswitch\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.377533 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377355 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.377533 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377328 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-run-ovn\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.377533 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377361 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-host-cni-bin\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.377533 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377396 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.377533 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377405 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-host-cni-netd\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.377533 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377421 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rdmc\" (UniqueName: \"kubernetes.io/projected/3253bdfe-5248-4d4c-9864-11660eb042ca-kube-api-access-5rdmc\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.377533 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:40.377444 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:40.377533 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377472 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-etc-kubernetes\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.377533 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:40.377506 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/971187a6-5709-48db-b990-0a13db9fad13-metrics-certs podName:971187a6-5709-48db-b990-0a13db9fad13 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:40.877486663 +0000 UTC m=+3.087216173 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/971187a6-5709-48db-b990-0a13db9fad13-metrics-certs") pod "network-metrics-daemon-zmkd8" (UID: "971187a6-5709-48db-b990-0a13db9fad13") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:40.377533 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377530 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-etc-kubernetes\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.377533 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377539 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-host-slash\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.378076 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377598 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-host-run-ovn-kubernetes\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.378076 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377627 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/056e740d-60e4-41f3-9f77-d5b292d6ef59-iptables-alerter-script\") pod \"iptables-alerter-pwvps\" (UID: \"056e740d-60e4-41f3-9f77-d5b292d6ef59\") " pod="openshift-network-operator/iptables-alerter-pwvps" Jul 06 23:48:40.378076 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377633 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-host-slash\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.378076 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377652 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-etc-tuned\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.378076 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377679 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-var-lib-openvswitch\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.378076 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377702 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3253bdfe-5248-4d4c-9864-11660eb042ca-ovnkube-config\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.378076 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377717 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-etc-sysctl-d\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.378076 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377728 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-var-lib-openvswitch\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.378076 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377732 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-etc-sysctl-conf\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.378076 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377777 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-host\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.378076 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377817 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-systemd-units\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.378076 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377865 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3253bdfe-5248-4d4c-9864-11660eb042ca-ovn-node-metrics-cert\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.378076 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377872 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-etc-sysctl-conf\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.378076 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377888 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-run\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.378736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377913 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3253bdfe-5248-4d4c-9864-11660eb042ca-env-overrides\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.378736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377921 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-systemd-units\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.378736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377940 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-node-log\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.378736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377963 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-lib-modules\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.378736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377966 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-host\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.378736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377678 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-host-run-ovn-kubernetes\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.378736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377982 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-etc-sysctl-d\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.378736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.377985 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-var-lib-kubelet\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.378736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378038 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/056e740d-60e4-41f3-9f77-d5b292d6ef59-host-slash\") pod \"iptables-alerter-pwvps\" (UID: \"056e740d-60e4-41f3-9f77-d5b292d6ef59\") " pod="openshift-network-operator/iptables-alerter-pwvps" Jul 06 23:48:40.378736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378042 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-node-log\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.378736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378075 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-host-kubelet\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.378736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378098 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-run-systemd\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.378736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378118 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-lib-modules\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.378736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378124 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3253bdfe-5248-4d4c-9864-11660eb042ca-ovnkube-script-lib\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.378736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378148 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-822d4\" (UniqueName: \"kubernetes.io/projected/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-kube-api-access-822d4\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.379270 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378172 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-run\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.379270 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378211 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/056e740d-60e4-41f3-9f77-d5b292d6ef59-host-slash\") pod \"iptables-alerter-pwvps\" (UID: \"056e740d-60e4-41f3-9f77-d5b292d6ef59\") " pod="openshift-network-operator/iptables-alerter-pwvps" Jul 06 23:48:40.379270 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378213 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-host-run-netns\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.379270 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378174 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-host-run-netns\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.379270 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378251 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-host-kubelet\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.379270 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378256 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdgbg\" (UniqueName: \"kubernetes.io/projected/971187a6-5709-48db-b990-0a13db9fad13-kube-api-access-pdgbg\") pod \"network-metrics-daemon-zmkd8\" (UID: \"971187a6-5709-48db-b990-0a13db9fad13\") " pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:48:40.379270 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378277 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/056e740d-60e4-41f3-9f77-d5b292d6ef59-iptables-alerter-script\") pod \"iptables-alerter-pwvps\" (UID: \"056e740d-60e4-41f3-9f77-d5b292d6ef59\") " pod="openshift-network-operator/iptables-alerter-pwvps" Jul 06 23:48:40.379270 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378287 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-run-systemd\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.379270 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378320 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-sys\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.379270 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378325 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3253bdfe-5248-4d4c-9864-11660eb042ca-ovnkube-config\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.379270 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378341 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3253bdfe-5248-4d4c-9864-11660eb042ca-env-overrides\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.379270 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378349 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-etc-sysconfig\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.379270 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378378 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-tmp\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.379270 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378391 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-var-lib-kubelet\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.379270 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378392 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-sys\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.379768 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378404 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h584q\" (UniqueName: \"kubernetes.io/projected/056e740d-60e4-41f3-9f77-d5b292d6ef59-kube-api-access-h584q\") pod \"iptables-alerter-pwvps\" (UID: \"056e740d-60e4-41f3-9f77-d5b292d6ef59\") " pod="openshift-network-operator/iptables-alerter-pwvps" Jul 06 23:48:40.379768 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378430 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-etc-sysconfig\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.379768 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378454 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-run-openvswitch\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.379768 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378486 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-log-socket\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.379768 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378509 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-etc-modprobe-d\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.379768 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378533 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-etc-systemd\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.379768 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378572 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-run-openvswitch\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.379768 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378611 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-etc-systemd\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.379768 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378641 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3253bdfe-5248-4d4c-9864-11660eb042ca-log-socket\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.379768 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378711 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-etc-modprobe-d\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.379768 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.378952 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3253bdfe-5248-4d4c-9864-11660eb042ca-ovnkube-script-lib\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.380613 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.380584 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3253bdfe-5248-4d4c-9864-11660eb042ca-ovn-node-metrics-cert\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.380723 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.380632 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-tmp\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.380723 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.380674 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-etc-tuned\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.387644 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.387620 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rdmc\" (UniqueName: \"kubernetes.io/projected/3253bdfe-5248-4d4c-9864-11660eb042ca-kube-api-access-5rdmc\") pod \"ovnkube-node-4z8vq\" (UID: \"3253bdfe-5248-4d4c-9864-11660eb042ca\") " pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.402557 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.402524 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-822d4\" (UniqueName: \"kubernetes.io/projected/3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963-kube-api-access-822d4\") pod \"tuned-8b784\" (UID: \"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963\") " pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.405495 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.405470 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdgbg\" (UniqueName: \"kubernetes.io/projected/971187a6-5709-48db-b990-0a13db9fad13-kube-api-access-pdgbg\") pod \"network-metrics-daemon-zmkd8\" (UID: \"971187a6-5709-48db-b990-0a13db9fad13\") " pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:48:40.405613 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.405470 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h584q\" (UniqueName: \"kubernetes.io/projected/056e740d-60e4-41f3-9f77-d5b292d6ef59-kube-api-access-h584q\") pod \"iptables-alerter-pwvps\" (UID: \"056e740d-60e4-41f3-9f77-d5b292d6ef59\") " pod="openshift-network-operator/iptables-alerter-pwvps" Jul 06 23:48:40.439500 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.439467 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2lwj" Jul 06 23:48:40.449007 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.448979 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6r2vp" Jul 06 23:48:40.457791 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.457766 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-p274v" Jul 06 23:48:40.465443 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.465421 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9tms8" Jul 06 23:48:40.475194 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.475170 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tdk4b" Jul 06 23:48:40.483009 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.482989 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j58wh" Jul 06 23:48:40.499778 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.499750 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pwvps" Jul 06 23:48:40.508737 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.508712 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8b784" Jul 06 23:48:40.515713 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.515685 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:48:40.781030 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.780950 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68c477a5-72a1-47ec-9864-17cd0021a5f9-original-pull-secret\") pod \"global-pull-secret-syncer-55rgj\" (UID: \"68c477a5-72a1-47ec-9864-17cd0021a5f9\") " pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:48:40.781182 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:40.781125 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:40.781238 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:40.781209 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68c477a5-72a1-47ec-9864-17cd0021a5f9-original-pull-secret podName:68c477a5-72a1-47ec-9864-17cd0021a5f9 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:41.78118782 +0000 UTC m=+3.990917322 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/68c477a5-72a1-47ec-9864-17cd0021a5f9-original-pull-secret") pod "global-pull-secret-syncer-55rgj" (UID: "68c477a5-72a1-47ec-9864-17cd0021a5f9") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:40.881872 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.881830 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp29w\" (UniqueName: \"kubernetes.io/projected/8f45c29d-2b22-4a11-bdc7-2e8dc27aab32-kube-api-access-sp29w\") pod \"network-check-target-qndcn\" (UID: \"8f45c29d-2b22-4a11-bdc7-2e8dc27aab32\") " pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:48:40.882044 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:40.881890 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/971187a6-5709-48db-b990-0a13db9fad13-metrics-certs\") pod \"network-metrics-daemon-zmkd8\" (UID: \"971187a6-5709-48db-b990-0a13db9fad13\") " pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:48:40.882044 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:40.882009 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:40.882118 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:40.882067 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/971187a6-5709-48db-b990-0a13db9fad13-metrics-certs podName:971187a6-5709-48db-b990-0a13db9fad13 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:41.882053151 +0000 UTC m=+4.091782650 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/971187a6-5709-48db-b990-0a13db9fad13-metrics-certs") pod "network-metrics-daemon-zmkd8" (UID: "971187a6-5709-48db-b990-0a13db9fad13") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:40.882118 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:40.882012 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:48:40.882118 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:40.882114 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:48:40.882216 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:40.882127 2553 projected.go:196] Error preparing data for projected volume kube-api-access-sp29w for pod openshift-network-diagnostics/network-check-target-qndcn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:40.882216 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:40.882171 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f45c29d-2b22-4a11-bdc7-2e8dc27aab32-kube-api-access-sp29w podName:8f45c29d-2b22-4a11-bdc7-2e8dc27aab32 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:41.88215888 +0000 UTC m=+4.091888383 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-sp29w" (UniqueName: "kubernetes.io/projected/8f45c29d-2b22-4a11-bdc7-2e8dc27aab32-kube-api-access-sp29w") pod "network-check-target-qndcn" (UID: "8f45c29d-2b22-4a11-bdc7-2e8dc27aab32") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:41.249113 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:41.248983 2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 23:43:40 +0000 UTC" deadline="2027-12-03 17:23:18.860625402 +0000 UTC" Jul 06 23:48:41.249113 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:41.249054 2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12353h34m37.611575499s" Jul 06 23:48:41.296580 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:41.296548 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:48:41.296735 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:41.296674 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qndcn" podUID="8f45c29d-2b22-4a11-bdc7-2e8dc27aab32" Jul 06 23:48:41.349951 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:41.349789 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3935a3b9_fbc6_4a9f_ad8a_9fb9866c0963.slice/crio-91970d71e5fd024e55dd08aaea0ef7b7dd436efd672825bca48a62e96c69883e WatchSource:0}: Error finding container 91970d71e5fd024e55dd08aaea0ef7b7dd436efd672825bca48a62e96c69883e: Status 404 returned error can't find the container with id 91970d71e5fd024e55dd08aaea0ef7b7dd436efd672825bca48a62e96c69883e Jul 06 23:48:41.351922 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:41.351850 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod991cd81c_10fa_4837_8cb9_e727879aa882.slice/crio-b1eb1fac3671bff085c7d1b752bd312ed77a6fd8aa4df2164be435e0e6ce8568 WatchSource:0}: Error finding container b1eb1fac3671bff085c7d1b752bd312ed77a6fd8aa4df2164be435e0e6ce8568: Status 404 returned error can't find the container with id b1eb1fac3671bff085c7d1b752bd312ed77a6fd8aa4df2164be435e0e6ce8568 Jul 06 23:48:41.355331 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:41.355245 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3253bdfe_5248_4d4c_9864_11660eb042ca.slice/crio-0f506488a20e5a841e66a82496b16121d720af3ab11c6e68a3dadd98c5f7e3a8 WatchSource:0}: Error finding container 0f506488a20e5a841e66a82496b16121d720af3ab11c6e68a3dadd98c5f7e3a8: Status 404 returned error can't find the container with id 0f506488a20e5a841e66a82496b16121d720af3ab11c6e68a3dadd98c5f7e3a8 Jul 06 23:48:41.355905 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:41.355882 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff6ba6b4_93ca_481f_8193_dfedf07c3875.slice/crio-578c8dad3db045e84cffab8d54fa99cb4cbbf733f1be3b54e6fac3ea71267f21 WatchSource:0}: Error finding container 578c8dad3db045e84cffab8d54fa99cb4cbbf733f1be3b54e6fac3ea71267f21: Status 404 returned error can't find the container with id 578c8dad3db045e84cffab8d54fa99cb4cbbf733f1be3b54e6fac3ea71267f21 Jul 06 23:48:41.357092 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:41.356395 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93308370_a961_4b85_a764_abac1d17c520.slice/crio-d1dc86f6acf1b22cba23b0e0ef67e0323f7f8a6d0e72b11fd37911645a15fa8f WatchSource:0}: Error finding container d1dc86f6acf1b22cba23b0e0ef67e0323f7f8a6d0e72b11fd37911645a15fa8f: Status 404 returned error can't find the container with id d1dc86f6acf1b22cba23b0e0ef67e0323f7f8a6d0e72b11fd37911645a15fa8f Jul 06 23:48:41.357520 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:41.357347 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f2c8799_a48e_4f4c_bce4_7f0187c24298.slice/crio-5f1285858ab9a68f4b7160f5bea8227c893934a63edcbe5f7ba9b5724f1ac60d WatchSource:0}: Error finding container 5f1285858ab9a68f4b7160f5bea8227c893934a63edcbe5f7ba9b5724f1ac60d: Status 404 returned error can't find the container with id 5f1285858ab9a68f4b7160f5bea8227c893934a63edcbe5f7ba9b5724f1ac60d Jul 06 23:48:41.358251 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:48:41.358124 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod056e740d_60e4_41f3_9f77_d5b292d6ef59.slice/crio-9366d9560b7fb531feb9821405c1a00ecbf4cabc17452703220252cbade6c808 WatchSource:0}: Error finding container 9366d9560b7fb531feb9821405c1a00ecbf4cabc17452703220252cbade6c808: Status 404 returned error can't find the container with id 9366d9560b7fb531feb9821405c1a00ecbf4cabc17452703220252cbade6c808 Jul 06 23:48:41.789023 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:41.788696 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68c477a5-72a1-47ec-9864-17cd0021a5f9-original-pull-secret\") pod \"global-pull-secret-syncer-55rgj\" (UID: \"68c477a5-72a1-47ec-9864-17cd0021a5f9\") " pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:48:41.789236 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:41.788980 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:41.789236 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:41.789148 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68c477a5-72a1-47ec-9864-17cd0021a5f9-original-pull-secret podName:68c477a5-72a1-47ec-9864-17cd0021a5f9 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:43.789127184 +0000 UTC m=+5.998856688 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/68c477a5-72a1-47ec-9864-17cd0021a5f9-original-pull-secret") pod "global-pull-secret-syncer-55rgj" (UID: "68c477a5-72a1-47ec-9864-17cd0021a5f9") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:41.889969 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:41.889879 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/971187a6-5709-48db-b990-0a13db9fad13-metrics-certs\") pod \"network-metrics-daemon-zmkd8\" (UID: \"971187a6-5709-48db-b990-0a13db9fad13\") " pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:48:41.889969 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:41.889956 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp29w\" (UniqueName: \"kubernetes.io/projected/8f45c29d-2b22-4a11-bdc7-2e8dc27aab32-kube-api-access-sp29w\") pod \"network-check-target-qndcn\" (UID: \"8f45c29d-2b22-4a11-bdc7-2e8dc27aab32\") " pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:48:41.890200 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:41.890104 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:48:41.890200 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:41.890122 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:48:41.890200 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:41.890135 2553 projected.go:196] Error preparing data for projected volume kube-api-access-sp29w for pod openshift-network-diagnostics/network-check-target-qndcn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:41.890200 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:41.890199 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f45c29d-2b22-4a11-bdc7-2e8dc27aab32-kube-api-access-sp29w podName:8f45c29d-2b22-4a11-bdc7-2e8dc27aab32 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:43.890180141 +0000 UTC m=+6.099909646 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-sp29w" (UniqueName: "kubernetes.io/projected/8f45c29d-2b22-4a11-bdc7-2e8dc27aab32-kube-api-access-sp29w") pod "network-check-target-qndcn" (UID: "8f45c29d-2b22-4a11-bdc7-2e8dc27aab32") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:41.890434 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:41.890272 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:41.890434 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:41.890322 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/971187a6-5709-48db-b990-0a13db9fad13-metrics-certs podName:971187a6-5709-48db-b990-0a13db9fad13 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:43.890308494 +0000 UTC m=+6.100037994 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/971187a6-5709-48db-b990-0a13db9fad13-metrics-certs") pod "network-metrics-daemon-zmkd8" (UID: "971187a6-5709-48db-b990-0a13db9fad13") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:42.250235 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:42.250085 2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 23:43:40 +0000 UTC" deadline="2028-01-13 06:53:15.338177087 +0000 UTC" Jul 06 23:48:42.250235 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:42.250130 2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13327h4m33.088052275s" Jul 06 23:48:42.298029 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:42.297424 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:48:42.298029 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:42.297559 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zmkd8" podUID="971187a6-5709-48db-b990-0a13db9fad13" Jul 06 23:48:42.298536 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:42.298358 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:48:42.298536 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:42.298477 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55rgj" podUID="68c477a5-72a1-47ec-9864-17cd0021a5f9" Jul 06 23:48:42.317372 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:42.316272 2553 generic.go:356] "Generic (PLEG): container finished" podID="2a2f7f2a1d78cf5fb862a37fcc75004a" containerID="80786d0b112912719109778effb99b1c1f587b4c7d87055924de6ca1c0460d4b" exitCode=0 Jul 06 23:48:42.317372 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:42.316379 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-25.ec2.internal" event={"ID":"2a2f7f2a1d78cf5fb862a37fcc75004a","Type":"ContainerDied","Data":"80786d0b112912719109778effb99b1c1f587b4c7d87055924de6ca1c0460d4b"} Jul 06 23:48:42.331697 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:42.331579 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-p274v" event={"ID":"ff9e3a5f-ea1a-4715-87b1-2766a82bef28","Type":"ContainerStarted","Data":"2502ee0e8d7f4b39d021b2cb4ae79de45b5429563d37a1ef60047be541827e17"} Jul 06 23:48:42.354326 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:42.354222 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6r2vp" event={"ID":"43908b50-c882-4e43-8d91-107f2acaf7b5","Type":"ContainerStarted","Data":"f64d113a29c922661b042c34d232bf1b9ca7ec152900b5341cf9921777b88515"} Jul 06 23:48:42.368419 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:42.368379 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pwvps" event={"ID":"056e740d-60e4-41f3-9f77-d5b292d6ef59","Type":"ContainerStarted","Data":"9366d9560b7fb531feb9821405c1a00ecbf4cabc17452703220252cbade6c808"} Jul 06 23:48:42.375358 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:42.370781 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" event={"ID":"3253bdfe-5248-4d4c-9864-11660eb042ca","Type":"ContainerStarted","Data":"0f506488a20e5a841e66a82496b16121d720af3ab11c6e68a3dadd98c5f7e3a8"} Jul 06 23:48:42.388541 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:42.388417 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2lwj" event={"ID":"991cd81c-10fa-4837-8cb9-e727879aa882","Type":"ContainerStarted","Data":"b1eb1fac3671bff085c7d1b752bd312ed77a6fd8aa4df2164be435e0e6ce8568"} Jul 06 23:48:42.399275 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:42.399183 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8b784" event={"ID":"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963","Type":"ContainerStarted","Data":"91970d71e5fd024e55dd08aaea0ef7b7dd436efd672825bca48a62e96c69883e"} Jul 06 23:48:42.429069 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:42.428991 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-25.ec2.internal" event={"ID":"ba668ff1926990fccf3f73e0625e305c","Type":"ContainerStarted","Data":"93e277fd793146f6907de053a91d9ed63b4cabc4a9bfb155ed5e0a74316e41b4"} Jul 06 23:48:42.449240 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:42.448804 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j58wh" event={"ID":"ff6ba6b4-93ca-481f-8193-dfedf07c3875","Type":"ContainerStarted","Data":"578c8dad3db045e84cffab8d54fa99cb4cbbf733f1be3b54e6fac3ea71267f21"} Jul 06 23:48:42.449714 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:42.449642 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-25.ec2.internal" podStartSLOduration=3.449621412 podStartE2EDuration="3.449621412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:48:42.448816436 +0000 UTC m=+4.658545985" watchObservedRunningTime="2026-07-06 23:48:42.449621412 +0000 UTC m=+4.659350935" Jul 06 23:48:42.454945 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:42.454427 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tdk4b" event={"ID":"4f2c8799-a48e-4f4c-bce4-7f0187c24298","Type":"ContainerStarted","Data":"5f1285858ab9a68f4b7160f5bea8227c893934a63edcbe5f7ba9b5724f1ac60d"} Jul 06 23:48:42.479923 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:42.479878 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9tms8" event={"ID":"93308370-a961-4b85-a764-abac1d17c520","Type":"ContainerStarted","Data":"d1dc86f6acf1b22cba23b0e0ef67e0323f7f8a6d0e72b11fd37911645a15fa8f"} Jul 06 23:48:43.296449 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:43.296415 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:48:43.296931 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:43.296549 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qndcn" podUID="8f45c29d-2b22-4a11-bdc7-2e8dc27aab32" Jul 06 23:48:43.510016 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:43.509905 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-25.ec2.internal" event={"ID":"2a2f7f2a1d78cf5fb862a37fcc75004a","Type":"ContainerStarted","Data":"d98934d215c4fc7650a800715fa349ab4899455d06cb79c962a0634d084bf573"} Jul 06 23:48:43.527529 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:43.527473 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-25.ec2.internal" podStartSLOduration=4.527452876 podStartE2EDuration="4.527452876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:48:43.527055785 +0000 UTC m=+5.736785306" watchObservedRunningTime="2026-07-06 23:48:43.527452876 +0000 UTC m=+5.737182400" Jul 06 23:48:43.810483 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:43.810436 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68c477a5-72a1-47ec-9864-17cd0021a5f9-original-pull-secret\") pod \"global-pull-secret-syncer-55rgj\" (UID: \"68c477a5-72a1-47ec-9864-17cd0021a5f9\") " pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:48:43.810662 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:43.810606 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:43.810737 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:43.810668 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68c477a5-72a1-47ec-9864-17cd0021a5f9-original-pull-secret podName:68c477a5-72a1-47ec-9864-17cd0021a5f9 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:47.810650153 +0000 UTC m=+10.020379654 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/68c477a5-72a1-47ec-9864-17cd0021a5f9-original-pull-secret") pod "global-pull-secret-syncer-55rgj" (UID: "68c477a5-72a1-47ec-9864-17cd0021a5f9") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:43.912774 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:43.911811 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/971187a6-5709-48db-b990-0a13db9fad13-metrics-certs\") pod \"network-metrics-daemon-zmkd8\" (UID: \"971187a6-5709-48db-b990-0a13db9fad13\") " pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:48:43.912774 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:43.911896 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp29w\" (UniqueName: \"kubernetes.io/projected/8f45c29d-2b22-4a11-bdc7-2e8dc27aab32-kube-api-access-sp29w\") pod \"network-check-target-qndcn\" (UID: \"8f45c29d-2b22-4a11-bdc7-2e8dc27aab32\") " pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:48:43.912774 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:43.912082 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:48:43.912774 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:43.912103 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:48:43.912774 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:43.912117 2553 projected.go:196] Error preparing data for projected volume kube-api-access-sp29w for pod openshift-network-diagnostics/network-check-target-qndcn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:43.912774 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:43.912185 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f45c29d-2b22-4a11-bdc7-2e8dc27aab32-kube-api-access-sp29w podName:8f45c29d-2b22-4a11-bdc7-2e8dc27aab32 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:47.912164502 +0000 UTC m=+10.121894023 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-sp29w" (UniqueName: "kubernetes.io/projected/8f45c29d-2b22-4a11-bdc7-2e8dc27aab32-kube-api-access-sp29w") pod "network-check-target-qndcn" (UID: "8f45c29d-2b22-4a11-bdc7-2e8dc27aab32") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:43.912774 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:43.912306 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:43.912774 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:43.912371 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/971187a6-5709-48db-b990-0a13db9fad13-metrics-certs podName:971187a6-5709-48db-b990-0a13db9fad13 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:47.912356114 +0000 UTC m=+10.122085615 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/971187a6-5709-48db-b990-0a13db9fad13-metrics-certs") pod "network-metrics-daemon-zmkd8" (UID: "971187a6-5709-48db-b990-0a13db9fad13") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:44.297907 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:44.297378 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:48:44.297907 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:44.297387 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:48:44.297907 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:44.297521 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55rgj" podUID="68c477a5-72a1-47ec-9864-17cd0021a5f9" Jul 06 23:48:44.297907 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:44.297616 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zmkd8" podUID="971187a6-5709-48db-b990-0a13db9fad13" Jul 06 23:48:45.297239 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:45.297203 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:48:45.297442 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:45.297372 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qndcn" podUID="8f45c29d-2b22-4a11-bdc7-2e8dc27aab32" Jul 06 23:48:46.297606 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:46.297046 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:48:46.297606 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:46.297195 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zmkd8" podUID="971187a6-5709-48db-b990-0a13db9fad13" Jul 06 23:48:46.298377 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:46.298202 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:48:46.298377 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:46.298335 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55rgj" podUID="68c477a5-72a1-47ec-9864-17cd0021a5f9" Jul 06 23:48:47.296762 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:47.296722 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:48:47.296978 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:47.296862 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qndcn" podUID="8f45c29d-2b22-4a11-bdc7-2e8dc27aab32" Jul 06 23:48:47.847626 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:47.847308 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68c477a5-72a1-47ec-9864-17cd0021a5f9-original-pull-secret\") pod \"global-pull-secret-syncer-55rgj\" (UID: \"68c477a5-72a1-47ec-9864-17cd0021a5f9\") " pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:48:47.847626 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:47.847504 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:47.847626 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:47.847609 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68c477a5-72a1-47ec-9864-17cd0021a5f9-original-pull-secret podName:68c477a5-72a1-47ec-9864-17cd0021a5f9 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:55.847588433 +0000 UTC m=+18.057317952 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/68c477a5-72a1-47ec-9864-17cd0021a5f9-original-pull-secret") pod "global-pull-secret-syncer-55rgj" (UID: "68c477a5-72a1-47ec-9864-17cd0021a5f9") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:47.948841 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:47.948204 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/971187a6-5709-48db-b990-0a13db9fad13-metrics-certs\") pod \"network-metrics-daemon-zmkd8\" (UID: \"971187a6-5709-48db-b990-0a13db9fad13\") " pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:48:47.948841 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:47.948284 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp29w\" (UniqueName: \"kubernetes.io/projected/8f45c29d-2b22-4a11-bdc7-2e8dc27aab32-kube-api-access-sp29w\") pod \"network-check-target-qndcn\" (UID: \"8f45c29d-2b22-4a11-bdc7-2e8dc27aab32\") " pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:48:47.948841 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:47.948431 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:47.948841 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:47.948448 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:48:47.948841 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:47.948467 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:48:47.948841 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:47.948479 2553 projected.go:196] Error preparing data for projected volume kube-api-access-sp29w for pod openshift-network-diagnostics/network-check-target-qndcn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:47.948841 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:47.948512 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/971187a6-5709-48db-b990-0a13db9fad13-metrics-certs podName:971187a6-5709-48db-b990-0a13db9fad13 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:55.948491278 +0000 UTC m=+18.158220790 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/971187a6-5709-48db-b990-0a13db9fad13-metrics-certs") pod "network-metrics-daemon-zmkd8" (UID: "971187a6-5709-48db-b990-0a13db9fad13") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:47.948841 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:47.948533 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f45c29d-2b22-4a11-bdc7-2e8dc27aab32-kube-api-access-sp29w podName:8f45c29d-2b22-4a11-bdc7-2e8dc27aab32 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:55.94852238 +0000 UTC m=+18.158251886 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-sp29w" (UniqueName: "kubernetes.io/projected/8f45c29d-2b22-4a11-bdc7-2e8dc27aab32-kube-api-access-sp29w") pod "network-check-target-qndcn" (UID: "8f45c29d-2b22-4a11-bdc7-2e8dc27aab32") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:48.298330 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:48.298199 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:48:48.298490 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:48.298340 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zmkd8" podUID="971187a6-5709-48db-b990-0a13db9fad13" Jul 06 23:48:48.298746 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:48.298207 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:48:48.298746 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:48.298683 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55rgj" podUID="68c477a5-72a1-47ec-9864-17cd0021a5f9" Jul 06 23:48:49.297248 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:49.297153 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:48:49.297696 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:49.297308 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qndcn" podUID="8f45c29d-2b22-4a11-bdc7-2e8dc27aab32" Jul 06 23:48:50.296959 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:50.296922 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:48:50.297134 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:50.297080 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zmkd8" podUID="971187a6-5709-48db-b990-0a13db9fad13" Jul 06 23:48:50.297247 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:50.297224 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:48:50.297382 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:50.297361 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55rgj" podUID="68c477a5-72a1-47ec-9864-17cd0021a5f9" Jul 06 23:48:51.296878 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:51.296840 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:48:51.297093 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:51.296992 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qndcn" podUID="8f45c29d-2b22-4a11-bdc7-2e8dc27aab32" Jul 06 23:48:52.297233 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:52.297190 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:48:52.297676 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:52.297190 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:48:52.297676 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:52.297343 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55rgj" podUID="68c477a5-72a1-47ec-9864-17cd0021a5f9" Jul 06 23:48:52.297676 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:52.297442 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zmkd8" podUID="971187a6-5709-48db-b990-0a13db9fad13" Jul 06 23:48:53.296950 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:53.296906 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:48:53.297128 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:53.297039 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qndcn" podUID="8f45c29d-2b22-4a11-bdc7-2e8dc27aab32" Jul 06 23:48:54.296995 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:54.296957 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:48:54.297449 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:54.296957 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:48:54.297449 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:54.297099 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55rgj" podUID="68c477a5-72a1-47ec-9864-17cd0021a5f9" Jul 06 23:48:54.297449 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:54.297147 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zmkd8" podUID="971187a6-5709-48db-b990-0a13db9fad13" Jul 06 23:48:55.297151 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:55.297112 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:48:55.297728 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:55.297252 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qndcn" podUID="8f45c29d-2b22-4a11-bdc7-2e8dc27aab32" Jul 06 23:48:55.911667 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:55.911629 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68c477a5-72a1-47ec-9864-17cd0021a5f9-original-pull-secret\") pod \"global-pull-secret-syncer-55rgj\" (UID: \"68c477a5-72a1-47ec-9864-17cd0021a5f9\") " pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:48:55.911855 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:55.911807 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:55.911902 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:55.911892 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68c477a5-72a1-47ec-9864-17cd0021a5f9-original-pull-secret podName:68c477a5-72a1-47ec-9864-17cd0021a5f9 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:11.91187143 +0000 UTC m=+34.121600933 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/68c477a5-72a1-47ec-9864-17cd0021a5f9-original-pull-secret") pod "global-pull-secret-syncer-55rgj" (UID: "68c477a5-72a1-47ec-9864-17cd0021a5f9") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:56.012584 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:56.012548 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/971187a6-5709-48db-b990-0a13db9fad13-metrics-certs\") pod \"network-metrics-daemon-zmkd8\" (UID: \"971187a6-5709-48db-b990-0a13db9fad13\") " pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:48:56.012790 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:56.012613 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp29w\" (UniqueName: \"kubernetes.io/projected/8f45c29d-2b22-4a11-bdc7-2e8dc27aab32-kube-api-access-sp29w\") pod \"network-check-target-qndcn\" (UID: \"8f45c29d-2b22-4a11-bdc7-2e8dc27aab32\") " pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:48:56.012790 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:56.012721 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:56.012790 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:56.012734 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:48:56.012790 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:56.012756 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:48:56.012790 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:56.012771 2553 projected.go:196] Error preparing data for projected volume kube-api-access-sp29w for pod openshift-network-diagnostics/network-check-target-qndcn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:56.013027 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:56.012796 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/971187a6-5709-48db-b990-0a13db9fad13-metrics-certs podName:971187a6-5709-48db-b990-0a13db9fad13 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:12.012774046 +0000 UTC m=+34.222503544 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/971187a6-5709-48db-b990-0a13db9fad13-metrics-certs") pod "network-metrics-daemon-zmkd8" (UID: "971187a6-5709-48db-b990-0a13db9fad13") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:56.013027 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:56.012824 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f45c29d-2b22-4a11-bdc7-2e8dc27aab32-kube-api-access-sp29w podName:8f45c29d-2b22-4a11-bdc7-2e8dc27aab32 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:12.012807762 +0000 UTC m=+34.222537271 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-sp29w" (UniqueName: "kubernetes.io/projected/8f45c29d-2b22-4a11-bdc7-2e8dc27aab32-kube-api-access-sp29w") pod "network-check-target-qndcn" (UID: "8f45c29d-2b22-4a11-bdc7-2e8dc27aab32") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:56.297328 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:56.297220 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:48:56.297328 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:56.297268 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:48:56.297770 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:56.297386 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zmkd8" podUID="971187a6-5709-48db-b990-0a13db9fad13" Jul 06 23:48:56.297770 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:56.297498 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55rgj" podUID="68c477a5-72a1-47ec-9864-17cd0021a5f9" Jul 06 23:48:57.297016 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:57.296975 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:48:57.297219 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:57.297099 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qndcn" podUID="8f45c29d-2b22-4a11-bdc7-2e8dc27aab32" Jul 06 23:48:58.297719 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:58.297681 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:48:58.297719 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:58.297710 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:48:58.298193 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:58.297795 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zmkd8" podUID="971187a6-5709-48db-b990-0a13db9fad13" Jul 06 23:48:58.298193 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:58.297991 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55rgj" podUID="68c477a5-72a1-47ec-9864-17cd0021a5f9" Jul 06 23:48:59.296582 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:48:59.296548 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:48:59.296770 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:48:59.296659 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qndcn" podUID="8f45c29d-2b22-4a11-bdc7-2e8dc27aab32" Jul 06 23:49:00.297289 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:00.296547 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:49:00.297289 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:00.296914 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zmkd8" podUID="971187a6-5709-48db-b990-0a13db9fad13" Jul 06 23:49:00.297289 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:00.296552 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:49:00.297818 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:00.297389 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55rgj" podUID="68c477a5-72a1-47ec-9864-17cd0021a5f9" Jul 06 23:49:00.547910 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:00.547869 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-p274v" event={"ID":"ff9e3a5f-ea1a-4715-87b1-2766a82bef28","Type":"ContainerStarted","Data":"1e35d2d7532eb1336389f88694a9369141d507f253d8bff46c42fa509cd0d2c0"} Jul 06 23:49:00.549673 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:00.549636 2553 generic.go:356] "Generic (PLEG): container finished" podID="43908b50-c882-4e43-8d91-107f2acaf7b5" containerID="ec502c034d76d4926318360c9421c7f93e9dab77ba000a3833ba77d3c4fb5a34" exitCode=0 Jul 06 23:49:00.549807 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:00.549744 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6r2vp" event={"ID":"43908b50-c882-4e43-8d91-107f2acaf7b5","Type":"ContainerDied","Data":"ec502c034d76d4926318360c9421c7f93e9dab77ba000a3833ba77d3c4fb5a34"} Jul 06 23:49:00.551478 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:00.551449 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2lwj" event={"ID":"991cd81c-10fa-4837-8cb9-e727879aa882","Type":"ContainerStarted","Data":"7cc7c15ed5d9709805e08ac325f9ad98f85dfcdaec03cecf7ddc4c2fb7a1c98f"} Jul 06 23:49:00.553485 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:00.553443 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8b784" event={"ID":"3935a3b9-fbc6-4a9f-ad8a-9fb9866c0963","Type":"ContainerStarted","Data":"22d85bf86ae475bc34791a19be99263e2089f1904e299f63bdfe1d500c1e2ba3"} Jul 06 23:49:00.554909 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:00.554886 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j58wh" event={"ID":"ff6ba6b4-93ca-481f-8193-dfedf07c3875","Type":"ContainerStarted","Data":"b81449ecd31122a0db06b27da8f8fa4b872cbd69612f507f1cda6c1c61c98d01"} Jul 06 23:49:00.556491 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:00.556468 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tdk4b" event={"ID":"4f2c8799-a48e-4f4c-bce4-7f0187c24298","Type":"ContainerStarted","Data":"b0927fa62111ce022e50a5bd2951ebfc91eba79f8a592796bce5179086ec9424"} Jul 06 23:49:00.557924 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:00.557881 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9tms8" event={"ID":"93308370-a961-4b85-a764-abac1d17c520","Type":"ContainerStarted","Data":"54b8a86bf379ff9ab9d65fd440a10db2358cdce80770e1df65fb29cc38e5b478"} Jul 06 23:49:00.564652 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:00.564600 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-p274v" podStartSLOduration=3.7920723450000002 podStartE2EDuration="22.564561693s" totalImagesPullingTime="18.772489348s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:00.564546917 +0000 UTC m=+22.774276461" watchObservedRunningTime="2026-07-06 23:49:00.564561693 +0000 UTC m=+22.774291230" Jul 06 23:49:00.580165 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:00.580116 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8b784" podStartSLOduration=3.8127087509999997 podStartE2EDuration="22.580099567s" totalImagesPullingTime="18.767390816s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:00.579770513 +0000 UTC m=+22.789500034" watchObservedRunningTime="2026-07-06 23:49:00.580099567 +0000 UTC m=+22.789829088" Jul 06 23:49:00.618391 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:00.618337 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-j58wh" podStartSLOduration=8.615592258 podStartE2EDuration="22.618324485s" totalImagesPullingTime="14.002732227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:00.617532629 +0000 UTC m=+22.827262151" watchObservedRunningTime="2026-07-06 23:49:00.618324485 +0000 UTC m=+22.828054006" Jul 06 23:49:00.639502 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:00.639453 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-9tms8" podStartSLOduration=8.63782545 podStartE2EDuration="22.639436951s" totalImagesPullingTime="14.001611501s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:00.639255385 +0000 UTC m=+22.848984906" watchObservedRunningTime="2026-07-06 23:49:00.639436951 +0000 UTC m=+22.849166471" Jul 06 23:49:00.652960 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:00.652910 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-tdk4b" podStartSLOduration=3.894358744 podStartE2EDuration="22.652894592s" totalImagesPullingTime="18.758535848s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:00.652802539 +0000 UTC m=+22.862532062" watchObservedRunningTime="2026-07-06 23:49:00.652894592 +0000 UTC m=+22.862624113" Jul 06 23:49:01.296824 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:01.296560 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:49:01.297026 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:01.296881 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qndcn" podUID="8f45c29d-2b22-4a11-bdc7-2e8dc27aab32" Jul 06 23:49:01.308744 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:01.308691 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-j58wh" Jul 06 23:49:01.309858 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:01.309823 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-j58wh" Jul 06 23:49:01.563311 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:01.563058 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pwvps" event={"ID":"056e740d-60e4-41f3-9f77-d5b292d6ef59","Type":"ContainerStarted","Data":"a7c21aaac9850ebaeddf21487b3e970d02a69a0e7b6b898fd5480a43742017d1"} Jul 06 23:49:01.565951 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:01.565926 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4z8vq_3253bdfe-5248-4d4c-9864-11660eb042ca/ovn-acl-logging/0.log" Jul 06 23:49:01.566310 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:01.566274 2553 generic.go:356] "Generic (PLEG): container finished" podID="3253bdfe-5248-4d4c-9864-11660eb042ca" containerID="89a3b715206d566684f8355a637237f52adb8aaf5270038f88d962b07363e303" exitCode=1 Jul 06 23:49:01.566407 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:01.566357 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" event={"ID":"3253bdfe-5248-4d4c-9864-11660eb042ca","Type":"ContainerStarted","Data":"1dff10cace9a6d6c006875214d02081c921127d3e2467118ff13957f95d49ce2"} Jul 06 23:49:01.566465 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:01.566423 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" event={"ID":"3253bdfe-5248-4d4c-9864-11660eb042ca","Type":"ContainerStarted","Data":"e54c472075a100769a3a35b5217995e53bf1d2933c3a521b1df34b3d3a69ca25"} Jul 06 23:49:01.566465 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:01.566459 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" event={"ID":"3253bdfe-5248-4d4c-9864-11660eb042ca","Type":"ContainerStarted","Data":"abbf255d82be4c31f3a5c92b379c6c2fdb2bbdbeb79e7ed35cf4ac0614fd7ee9"} Jul 06 23:49:01.566563 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:01.566489 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" event={"ID":"3253bdfe-5248-4d4c-9864-11660eb042ca","Type":"ContainerStarted","Data":"8befc422a8e6b3aa2a9892efbc4d2333521ea7836cec5466a7f0d17f2a374815"} Jul 06 23:49:01.566563 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:01.566536 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" event={"ID":"3253bdfe-5248-4d4c-9864-11660eb042ca","Type":"ContainerDied","Data":"89a3b715206d566684f8355a637237f52adb8aaf5270038f88d962b07363e303"} Jul 06 23:49:01.566659 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:01.566588 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" event={"ID":"3253bdfe-5248-4d4c-9864-11660eb042ca","Type":"ContainerStarted","Data":"9511cb794e607b17bf7da5b509317edd97235229bbe2f4f622a7e4475bf6b8ad"} Jul 06 23:49:01.566659 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:01.566640 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-j58wh" Jul 06 23:49:01.567082 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:01.567065 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-j58wh" Jul 06 23:49:01.577574 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:01.577521 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-pwvps" podStartSLOduration=4.838297911 podStartE2EDuration="23.577501207s" totalImagesPullingTime="18.739203296s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:01.576854594 +0000 UTC m=+23.786584116" watchObservedRunningTime="2026-07-06 23:49:01.577501207 +0000 UTC m=+23.787230729" Jul 06 23:49:01.676475 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:01.676434 2553 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 23:49:02.269142 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:02.269014 2553 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T23:49:01.6764614Z","UUID":"e1c0ceb9-7b29-4340-af9f-9e889befe3a5","Handler":null,"Name":"","Endpoint":""} Jul 06 23:49:02.270997 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:02.270713 2553 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 23:49:02.270997 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:02.270759 2553 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 23:49:02.296890 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:02.296332 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:49:02.296890 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:02.296377 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:49:02.296890 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:02.296469 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zmkd8" podUID="971187a6-5709-48db-b990-0a13db9fad13" Jul 06 23:49:02.296890 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:02.296586 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55rgj" podUID="68c477a5-72a1-47ec-9864-17cd0021a5f9" Jul 06 23:49:02.572016 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:02.571921 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2lwj" event={"ID":"991cd81c-10fa-4837-8cb9-e727879aa882","Type":"ContainerStarted","Data":"50663352a885baee1615752431fdabc2e37539f2ec158084cdf04ac02759bedf"} Jul 06 23:49:03.296951 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:03.296736 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:49:03.297086 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:03.297063 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qndcn" podUID="8f45c29d-2b22-4a11-bdc7-2e8dc27aab32" Jul 06 23:49:03.578208 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:03.578135 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4z8vq_3253bdfe-5248-4d4c-9864-11660eb042ca/ovn-acl-logging/0.log" Jul 06 23:49:03.578657 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:03.578596 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" event={"ID":"3253bdfe-5248-4d4c-9864-11660eb042ca","Type":"ContainerStarted","Data":"fad74b249e89a6cbad1458d1cc764cca36c83e08a77493fe958ffd55e9068d06"} Jul 06 23:49:03.581105 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:03.581064 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2lwj" event={"ID":"991cd81c-10fa-4837-8cb9-e727879aa882","Type":"ContainerStarted","Data":"db30aee7a1088f770b316183c9dd15d64068fba17b76700198de90bb556cc9f1"} Jul 06 23:49:03.601042 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:03.600993 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-t2lwj" podStartSLOduration=4.65282923 podStartE2EDuration="25.600975268s" totalImagesPullingTime="20.948146038s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:03.60089939 +0000 UTC m=+25.810628912" watchObservedRunningTime="2026-07-06 23:49:03.600975268 +0000 UTC m=+25.810704779" Jul 06 23:49:04.297142 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:04.297101 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:49:04.297405 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:04.297149 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:49:04.297405 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:04.297259 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zmkd8" podUID="971187a6-5709-48db-b990-0a13db9fad13" Jul 06 23:49:04.297405 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:04.297391 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55rgj" podUID="68c477a5-72a1-47ec-9864-17cd0021a5f9" Jul 06 23:49:05.297404 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:05.297373 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:49:05.298028 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:05.297504 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qndcn" podUID="8f45c29d-2b22-4a11-bdc7-2e8dc27aab32" Jul 06 23:49:06.300425 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:06.300161 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:49:06.300952 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:06.300161 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:49:06.300952 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:06.300451 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55rgj" podUID="68c477a5-72a1-47ec-9864-17cd0021a5f9" Jul 06 23:49:06.300952 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:06.300537 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zmkd8" podUID="971187a6-5709-48db-b990-0a13db9fad13" Jul 06 23:49:06.589691 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:06.589605 2553 generic.go:356] "Generic (PLEG): container finished" podID="43908b50-c882-4e43-8d91-107f2acaf7b5" containerID="c41d23bc2a3cc3d0a59c37af40cd1f650e61bc6ff45b79189a4f71a3ece95295" exitCode=0 Jul 06 23:49:06.589853 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:06.589697 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6r2vp" event={"ID":"43908b50-c882-4e43-8d91-107f2acaf7b5","Type":"ContainerDied","Data":"c41d23bc2a3cc3d0a59c37af40cd1f650e61bc6ff45b79189a4f71a3ece95295"} Jul 06 23:49:06.593489 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:06.593466 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4z8vq_3253bdfe-5248-4d4c-9864-11660eb042ca/ovn-acl-logging/0.log" Jul 06 23:49:06.593829 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:06.593804 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" event={"ID":"3253bdfe-5248-4d4c-9864-11660eb042ca","Type":"ContainerStarted","Data":"a74796a131afd2ae333890468b06ffbff7f502f35b93d74a90cdb374dd296626"} Jul 06 23:49:06.594077 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:06.594062 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:49:06.594153 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:06.594103 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:49:06.594276 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:06.594260 2553 scope.go:117] "RemoveContainer" containerID="89a3b715206d566684f8355a637237f52adb8aaf5270038f88d962b07363e303" Jul 06 23:49:06.609526 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:06.609503 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:49:06.609655 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:06.609606 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:49:07.296910 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:07.296876 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:49:07.297166 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:07.296983 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qndcn" podUID="8f45c29d-2b22-4a11-bdc7-2e8dc27aab32" Jul 06 23:49:07.601408 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:07.601377 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4z8vq_3253bdfe-5248-4d4c-9864-11660eb042ca/ovn-acl-logging/0.log" Jul 06 23:49:07.601897 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:07.601855 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" event={"ID":"3253bdfe-5248-4d4c-9864-11660eb042ca","Type":"ContainerStarted","Data":"9dde3f50cd39825f140efdee538cb35ebfe0dbd0e82dc213ddfe34745b04afa5"} Jul 06 23:49:07.602017 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:07.601971 2553 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 23:49:07.643496 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:07.643445 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" podStartSLOduration=10.512761923 podStartE2EDuration="29.643428439s" totalImagesPullingTime="19.130666516s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:07.642742068 +0000 UTC m=+29.852471610" watchObservedRunningTime="2026-07-06 23:49:07.643428439 +0000 UTC m=+29.853157963" Jul 06 23:49:07.793410 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:07.793381 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-55rgj"] Jul 06 23:49:07.793571 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:07.793493 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:49:07.793615 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:07.793577 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55rgj" podUID="68c477a5-72a1-47ec-9864-17cd0021a5f9" Jul 06 23:49:07.797286 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:07.797257 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qndcn"] Jul 06 23:49:07.797460 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:07.797395 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:49:07.797523 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:07.797497 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qndcn" podUID="8f45c29d-2b22-4a11-bdc7-2e8dc27aab32" Jul 06 23:49:07.797950 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:07.797921 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zmkd8"] Jul 06 23:49:07.798087 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:07.798035 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:49:07.798150 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:07.798117 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zmkd8" podUID="971187a6-5709-48db-b990-0a13db9fad13" Jul 06 23:49:08.606343 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:08.606291 2553 generic.go:356] "Generic (PLEG): container finished" podID="43908b50-c882-4e43-8d91-107f2acaf7b5" containerID="70fc6a1c3d8d68c00de155520706abd8aa4013fef6d28b85cd49b117b80d4496" exitCode=0 Jul 06 23:49:08.606737 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:08.606390 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6r2vp" event={"ID":"43908b50-c882-4e43-8d91-107f2acaf7b5","Type":"ContainerDied","Data":"70fc6a1c3d8d68c00de155520706abd8aa4013fef6d28b85cd49b117b80d4496"} Jul 06 23:49:08.606737 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:08.606502 2553 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 23:49:08.939616 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:08.939582 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:49:09.297027 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:09.296798 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:49:09.297180 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:09.296838 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:49:09.297180 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:09.297061 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55rgj" podUID="68c477a5-72a1-47ec-9864-17cd0021a5f9" Jul 06 23:49:09.297180 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:09.296838 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:49:09.297180 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:09.297135 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qndcn" podUID="8f45c29d-2b22-4a11-bdc7-2e8dc27aab32" Jul 06 23:49:09.297342 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:09.297206 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zmkd8" podUID="971187a6-5709-48db-b990-0a13db9fad13" Jul 06 23:49:10.614128 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:10.614085 2553 generic.go:356] "Generic (PLEG): container finished" podID="43908b50-c882-4e43-8d91-107f2acaf7b5" containerID="e801246a0775678d56ff63d03ed11c6925b70eb6879b69693b25ec3926fdc9a5" exitCode=0 Jul 06 23:49:10.614773 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:10.614166 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6r2vp" event={"ID":"43908b50-c882-4e43-8d91-107f2acaf7b5","Type":"ContainerDied","Data":"e801246a0775678d56ff63d03ed11c6925b70eb6879b69693b25ec3926fdc9a5"} Jul 06 23:49:11.296391 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:11.296357 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:49:11.296539 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:11.296357 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:49:11.296539 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:11.296500 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55rgj" podUID="68c477a5-72a1-47ec-9864-17cd0021a5f9" Jul 06 23:49:11.296651 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:11.296356 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:49:11.296651 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:11.296558 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zmkd8" podUID="971187a6-5709-48db-b990-0a13db9fad13" Jul 06 23:49:11.296740 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:11.296647 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qndcn" podUID="8f45c29d-2b22-4a11-bdc7-2e8dc27aab32" Jul 06 23:49:11.927349 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:11.927290 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68c477a5-72a1-47ec-9864-17cd0021a5f9-original-pull-secret\") pod \"global-pull-secret-syncer-55rgj\" (UID: \"68c477a5-72a1-47ec-9864-17cd0021a5f9\") " pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:49:11.927736 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:11.927447 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:49:11.927736 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:11.927511 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68c477a5-72a1-47ec-9864-17cd0021a5f9-original-pull-secret podName:68c477a5-72a1-47ec-9864-17cd0021a5f9 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:43.927495427 +0000 UTC m=+66.137224927 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/68c477a5-72a1-47ec-9864-17cd0021a5f9-original-pull-secret") pod "global-pull-secret-syncer-55rgj" (UID: "68c477a5-72a1-47ec-9864-17cd0021a5f9") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:49:12.027956 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:12.027902 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/971187a6-5709-48db-b990-0a13db9fad13-metrics-certs\") pod \"network-metrics-daemon-zmkd8\" (UID: \"971187a6-5709-48db-b990-0a13db9fad13\") " pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:49:12.028146 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:12.027978 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp29w\" (UniqueName: \"kubernetes.io/projected/8f45c29d-2b22-4a11-bdc7-2e8dc27aab32-kube-api-access-sp29w\") pod \"network-check-target-qndcn\" (UID: \"8f45c29d-2b22-4a11-bdc7-2e8dc27aab32\") " pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:49:12.028146 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:12.028074 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:49:12.028146 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:12.028124 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:49:12.028274 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:12.028148 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:49:12.028274 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:12.028156 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/971187a6-5709-48db-b990-0a13db9fad13-metrics-certs podName:971187a6-5709-48db-b990-0a13db9fad13 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:44.028135594 +0000 UTC m=+66.237865113 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/971187a6-5709-48db-b990-0a13db9fad13-metrics-certs") pod "network-metrics-daemon-zmkd8" (UID: "971187a6-5709-48db-b990-0a13db9fad13") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:49:12.028274 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:12.028163 2553 projected.go:196] Error preparing data for projected volume kube-api-access-sp29w for pod openshift-network-diagnostics/network-check-target-qndcn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:49:12.028274 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:12.028213 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8f45c29d-2b22-4a11-bdc7-2e8dc27aab32-kube-api-access-sp29w podName:8f45c29d-2b22-4a11-bdc7-2e8dc27aab32 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:44.028196777 +0000 UTC m=+66.237926282 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-sp29w" (UniqueName: "kubernetes.io/projected/8f45c29d-2b22-4a11-bdc7-2e8dc27aab32-kube-api-access-sp29w") pod "network-check-target-qndcn" (UID: "8f45c29d-2b22-4a11-bdc7-2e8dc27aab32") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:49:13.170394 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.170364 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-25.ec2.internal" event="NodeReady" Jul 06 23:49:13.170874 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.170535 2553 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 23:49:13.205030 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.205000 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7c586d98b6-vr5mc"] Jul 06 23:49:13.208941 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.208924 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:13.211455 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.211414 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 23:49:13.211739 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.211715 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 23:49:13.211739 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.211727 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 23:49:13.211880 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.211805 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-csfd5\"" Jul 06 23:49:13.218234 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.218208 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 23:49:13.222419 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.222395 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7gntn"] Jul 06 23:49:13.227116 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.227094 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c586d98b6-vr5mc"] Jul 06 23:49:13.227223 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.227127 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-v978g"] Jul 06 23:49:13.227258 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.227216 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7gntn" Jul 06 23:49:13.229838 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.229816 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 23:49:13.229971 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.229951 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-w55hv\"" Jul 06 23:49:13.230064 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.230050 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 23:49:13.230122 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.230086 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 23:49:13.231053 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.231038 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v978g" Jul 06 23:49:13.233426 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.233409 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 23:49:13.233536 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.233466 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-k5jpj\"" Jul 06 23:49:13.233651 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.233640 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 23:49:13.235829 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.235812 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7gntn"] Jul 06 23:49:13.239727 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.239490 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-v978g"] Jul 06 23:49:13.297006 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.296974 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:49:13.297193 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.296977 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:49:13.297193 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.296977 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:49:13.299620 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.299601 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 23:49:13.299844 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.299674 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nzg9j\"" Jul 06 23:49:13.299960 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.299886 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 23:49:13.299960 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.299902 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 23:49:13.299960 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.299694 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dwstx\"" Jul 06 23:49:13.300181 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.300163 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 23:49:13.337696 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.337665 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-tls\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:13.337696 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.337700 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-installation-pull-secrets\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:13.337935 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.337719 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnwkq\" (UniqueName: \"kubernetes.io/projected/e7b30a60-5ac4-43d8-891c-d23dd770b332-kube-api-access-xnwkq\") pod \"ingress-canary-7gntn\" (UID: \"e7b30a60-5ac4-43d8-891c-d23dd770b332\") " pod="openshift-ingress-canary/ingress-canary-7gntn" Jul 06 23:49:13.337935 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.337746 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvmc8\" (UniqueName: \"kubernetes.io/projected/05a76f14-b4fd-4087-bd7a-53587b38ea81-kube-api-access-lvmc8\") pod \"dns-default-v978g\" (UID: \"05a76f14-b4fd-4087-bd7a-53587b38ea81\") " pod="openshift-dns/dns-default-v978g" Jul 06 23:49:13.337935 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.337817 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj4tb\" (UniqueName: \"kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-kube-api-access-jj4tb\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:13.337935 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.337865 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-image-registry-private-configuration\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:13.337935 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.337903 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-bound-sa-token\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:13.338170 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.338025 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-certificates\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:13.338170 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.338059 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-trusted-ca\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:13.338170 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.338104 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-ca-trust-extracted\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:13.338316 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.338171 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e7b30a60-5ac4-43d8-891c-d23dd770b332-cert\") pod \"ingress-canary-7gntn\" (UID: \"e7b30a60-5ac4-43d8-891c-d23dd770b332\") " pod="openshift-ingress-canary/ingress-canary-7gntn" Jul 06 23:49:13.338316 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.338215 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/05a76f14-b4fd-4087-bd7a-53587b38ea81-config-volume\") pod \"dns-default-v978g\" (UID: \"05a76f14-b4fd-4087-bd7a-53587b38ea81\") " pod="openshift-dns/dns-default-v978g" Jul 06 23:49:13.338316 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.338238 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/05a76f14-b4fd-4087-bd7a-53587b38ea81-metrics-tls\") pod \"dns-default-v978g\" (UID: \"05a76f14-b4fd-4087-bd7a-53587b38ea81\") " pod="openshift-dns/dns-default-v978g" Jul 06 23:49:13.338316 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.338266 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05a76f14-b4fd-4087-bd7a-53587b38ea81-tmp-dir\") pod \"dns-default-v978g\" (UID: \"05a76f14-b4fd-4087-bd7a-53587b38ea81\") " pod="openshift-dns/dns-default-v978g" Jul 06 23:49:13.438978 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.438889 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj4tb\" (UniqueName: \"kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-kube-api-access-jj4tb\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:13.438978 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.438947 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-image-registry-private-configuration\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:13.439209 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.439075 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-bound-sa-token\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:13.439209 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.439159 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-certificates\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:13.439209 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.439187 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-trusted-ca\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:13.439379 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.439228 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-ca-trust-extracted\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:13.439379 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.439260 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e7b30a60-5ac4-43d8-891c-d23dd770b332-cert\") pod \"ingress-canary-7gntn\" (UID: \"e7b30a60-5ac4-43d8-891c-d23dd770b332\") " pod="openshift-ingress-canary/ingress-canary-7gntn" Jul 06 23:49:13.439379 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.439281 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/05a76f14-b4fd-4087-bd7a-53587b38ea81-config-volume\") pod \"dns-default-v978g\" (UID: \"05a76f14-b4fd-4087-bd7a-53587b38ea81\") " pod="openshift-dns/dns-default-v978g" Jul 06 23:49:13.439671 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.439645 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/05a76f14-b4fd-4087-bd7a-53587b38ea81-metrics-tls\") pod \"dns-default-v978g\" (UID: \"05a76f14-b4fd-4087-bd7a-53587b38ea81\") " pod="openshift-dns/dns-default-v978g" Jul 06 23:49:13.439773 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.439692 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05a76f14-b4fd-4087-bd7a-53587b38ea81-tmp-dir\") pod \"dns-default-v978g\" (UID: \"05a76f14-b4fd-4087-bd7a-53587b38ea81\") " pod="openshift-dns/dns-default-v978g" Jul 06 23:49:13.439773 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.439724 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-tls\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:13.439773 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.439743 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-ca-trust-extracted\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:13.439937 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:13.439831 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 23:49:13.439937 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.439844 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/05a76f14-b4fd-4087-bd7a-53587b38ea81-config-volume\") pod \"dns-default-v978g\" (UID: \"05a76f14-b4fd-4087-bd7a-53587b38ea81\") " pod="openshift-dns/dns-default-v978g" Jul 06 23:49:13.439937 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:13.439891 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05a76f14-b4fd-4087-bd7a-53587b38ea81-metrics-tls podName:05a76f14-b4fd-4087-bd7a-53587b38ea81 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:13.939872745 +0000 UTC m=+36.149602264 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/05a76f14-b4fd-4087-bd7a-53587b38ea81-metrics-tls") pod "dns-default-v978g" (UID: "05a76f14-b4fd-4087-bd7a-53587b38ea81") : secret "dns-default-metrics-tls" not found Jul 06 23:49:13.440390 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.440091 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05a76f14-b4fd-4087-bd7a-53587b38ea81-tmp-dir\") pod \"dns-default-v978g\" (UID: \"05a76f14-b4fd-4087-bd7a-53587b38ea81\") " pod="openshift-dns/dns-default-v978g" Jul 06 23:49:13.440390 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.439750 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-installation-pull-secrets\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:13.440390 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.440149 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnwkq\" (UniqueName: \"kubernetes.io/projected/e7b30a60-5ac4-43d8-891c-d23dd770b332-kube-api-access-xnwkq\") pod \"ingress-canary-7gntn\" (UID: \"e7b30a60-5ac4-43d8-891c-d23dd770b332\") " pod="openshift-ingress-canary/ingress-canary-7gntn" Jul 06 23:49:13.440390 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.440173 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvmc8\" (UniqueName: \"kubernetes.io/projected/05a76f14-b4fd-4087-bd7a-53587b38ea81-kube-api-access-lvmc8\") pod \"dns-default-v978g\" (UID: \"05a76f14-b4fd-4087-bd7a-53587b38ea81\") " pod="openshift-dns/dns-default-v978g" Jul 06 23:49:13.440390 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:13.440175 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 23:49:13.440390 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:13.440191 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c586d98b6-vr5mc: secret "image-registry-tls" not found Jul 06 23:49:13.440390 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:13.440235 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-tls podName:1bc2dc5e-4eab-40a3-9720-ff2ae2746cad nodeName:}" failed. No retries permitted until 2026-07-06 23:49:13.940219325 +0000 UTC m=+36.149948851 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-tls") pod "image-registry-7c586d98b6-vr5mc" (UID: "1bc2dc5e-4eab-40a3-9720-ff2ae2746cad") : secret "image-registry-tls" not found Jul 06 23:49:13.440753 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.440529 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-trusted-ca\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:13.441059 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.440879 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-certificates\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:13.443929 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.443858 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e7b30a60-5ac4-43d8-891c-d23dd770b332-cert\") pod \"ingress-canary-7gntn\" (UID: \"e7b30a60-5ac4-43d8-891c-d23dd770b332\") " pod="openshift-ingress-canary/ingress-canary-7gntn" Jul 06 23:49:13.443929 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.443889 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-image-registry-private-configuration\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:13.443929 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.443913 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-installation-pull-secrets\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:13.448999 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.448974 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-bound-sa-token\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:13.449164 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.449005 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvmc8\" (UniqueName: \"kubernetes.io/projected/05a76f14-b4fd-4087-bd7a-53587b38ea81-kube-api-access-lvmc8\") pod \"dns-default-v978g\" (UID: \"05a76f14-b4fd-4087-bd7a-53587b38ea81\") " pod="openshift-dns/dns-default-v978g" Jul 06 23:49:13.449164 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.449033 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj4tb\" (UniqueName: \"kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-kube-api-access-jj4tb\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:13.449164 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.449080 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnwkq\" (UniqueName: \"kubernetes.io/projected/e7b30a60-5ac4-43d8-891c-d23dd770b332-kube-api-access-xnwkq\") pod \"ingress-canary-7gntn\" (UID: \"e7b30a60-5ac4-43d8-891c-d23dd770b332\") " pod="openshift-ingress-canary/ingress-canary-7gntn" Jul 06 23:49:13.537805 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.537773 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7gntn" Jul 06 23:49:13.704909 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.704814 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7gntn"] Jul 06 23:49:13.708793 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:49:13.708762 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7b30a60_5ac4_43d8_891c_d23dd770b332.slice/crio-698b44571f4dd88e8dd3771515fac184e040ca67c55ec79c979bc805ff1155d0 WatchSource:0}: Error finding container 698b44571f4dd88e8dd3771515fac184e040ca67c55ec79c979bc805ff1155d0: Status 404 returned error can't find the container with id 698b44571f4dd88e8dd3771515fac184e040ca67c55ec79c979bc805ff1155d0 Jul 06 23:49:13.945125 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.945089 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/05a76f14-b4fd-4087-bd7a-53587b38ea81-metrics-tls\") pod \"dns-default-v978g\" (UID: \"05a76f14-b4fd-4087-bd7a-53587b38ea81\") " pod="openshift-dns/dns-default-v978g" Jul 06 23:49:13.945363 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:13.945134 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-tls\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:13.945363 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:13.945253 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 23:49:13.945363 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:13.945265 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c586d98b6-vr5mc: secret "image-registry-tls" not found Jul 06 23:49:13.945363 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:13.945265 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 23:49:13.945363 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:13.945343 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-tls podName:1bc2dc5e-4eab-40a3-9720-ff2ae2746cad nodeName:}" failed. No retries permitted until 2026-07-06 23:49:14.94532919 +0000 UTC m=+37.155058689 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-tls") pod "image-registry-7c586d98b6-vr5mc" (UID: "1bc2dc5e-4eab-40a3-9720-ff2ae2746cad") : secret "image-registry-tls" not found Jul 06 23:49:13.945363 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:13.945357 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05a76f14-b4fd-4087-bd7a-53587b38ea81-metrics-tls podName:05a76f14-b4fd-4087-bd7a-53587b38ea81 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:14.945350697 +0000 UTC m=+37.155080197 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/05a76f14-b4fd-4087-bd7a-53587b38ea81-metrics-tls") pod "dns-default-v978g" (UID: "05a76f14-b4fd-4087-bd7a-53587b38ea81") : secret "dns-default-metrics-tls" not found Jul 06 23:49:14.628498 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:14.628459 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7gntn" event={"ID":"e7b30a60-5ac4-43d8-891c-d23dd770b332","Type":"ContainerStarted","Data":"698b44571f4dd88e8dd3771515fac184e040ca67c55ec79c979bc805ff1155d0"} Jul 06 23:49:14.951762 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:14.951716 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/05a76f14-b4fd-4087-bd7a-53587b38ea81-metrics-tls\") pod \"dns-default-v978g\" (UID: \"05a76f14-b4fd-4087-bd7a-53587b38ea81\") " pod="openshift-dns/dns-default-v978g" Jul 06 23:49:14.951762 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:14.951764 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-tls\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:14.952010 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:14.951876 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 23:49:14.952010 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:14.951915 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 23:49:14.952010 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:14.951936 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c586d98b6-vr5mc: secret "image-registry-tls" not found Jul 06 23:49:14.952010 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:14.951959 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05a76f14-b4fd-4087-bd7a-53587b38ea81-metrics-tls podName:05a76f14-b4fd-4087-bd7a-53587b38ea81 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:16.951938506 +0000 UTC m=+39.161668005 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/05a76f14-b4fd-4087-bd7a-53587b38ea81-metrics-tls") pod "dns-default-v978g" (UID: "05a76f14-b4fd-4087-bd7a-53587b38ea81") : secret "dns-default-metrics-tls" not found Jul 06 23:49:14.952010 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:14.951989 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-tls podName:1bc2dc5e-4eab-40a3-9720-ff2ae2746cad nodeName:}" failed. No retries permitted until 2026-07-06 23:49:16.9519711 +0000 UTC m=+39.161700606 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-tls") pod "image-registry-7c586d98b6-vr5mc" (UID: "1bc2dc5e-4eab-40a3-9720-ff2ae2746cad") : secret "image-registry-tls" not found Jul 06 23:49:15.634073 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:15.633994 2553 generic.go:356] "Generic (PLEG): container finished" podID="43908b50-c882-4e43-8d91-107f2acaf7b5" containerID="54d31b67fb9bc567c2eb61e9b03faeb7cda1faa399c61af1cc489d11ba3b38ac" exitCode=0 Jul 06 23:49:15.634073 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:15.634042 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6r2vp" event={"ID":"43908b50-c882-4e43-8d91-107f2acaf7b5","Type":"ContainerDied","Data":"54d31b67fb9bc567c2eb61e9b03faeb7cda1faa399c61af1cc489d11ba3b38ac"} Jul 06 23:49:16.640584 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:16.640308 2553 generic.go:356] "Generic (PLEG): container finished" podID="43908b50-c882-4e43-8d91-107f2acaf7b5" containerID="4c26567d1d9884fd402c33f82d847ba6e9bb6958498133dd53bbc5c987c329bd" exitCode=0 Jul 06 23:49:16.641033 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:16.640391 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6r2vp" event={"ID":"43908b50-c882-4e43-8d91-107f2acaf7b5","Type":"ContainerDied","Data":"4c26567d1d9884fd402c33f82d847ba6e9bb6958498133dd53bbc5c987c329bd"} Jul 06 23:49:16.973449 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:16.973395 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/05a76f14-b4fd-4087-bd7a-53587b38ea81-metrics-tls\") pod \"dns-default-v978g\" (UID: \"05a76f14-b4fd-4087-bd7a-53587b38ea81\") " pod="openshift-dns/dns-default-v978g" Jul 06 23:49:16.973449 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:16.973451 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-tls\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:16.973716 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:16.973566 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 23:49:16.973716 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:16.973642 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05a76f14-b4fd-4087-bd7a-53587b38ea81-metrics-tls podName:05a76f14-b4fd-4087-bd7a-53587b38ea81 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:20.973620217 +0000 UTC m=+43.183349722 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/05a76f14-b4fd-4087-bd7a-53587b38ea81-metrics-tls") pod "dns-default-v978g" (UID: "05a76f14-b4fd-4087-bd7a-53587b38ea81") : secret "dns-default-metrics-tls" not found Jul 06 23:49:16.973716 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:16.973569 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 23:49:16.973716 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:16.973688 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c586d98b6-vr5mc: secret "image-registry-tls" not found Jul 06 23:49:16.973910 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:16.973767 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-tls podName:1bc2dc5e-4eab-40a3-9720-ff2ae2746cad nodeName:}" failed. No retries permitted until 2026-07-06 23:49:20.973747527 +0000 UTC m=+43.183477041 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-tls") pod "image-registry-7c586d98b6-vr5mc" (UID: "1bc2dc5e-4eab-40a3-9720-ff2ae2746cad") : secret "image-registry-tls" not found Jul 06 23:49:17.645920 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:17.645883 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6r2vp" event={"ID":"43908b50-c882-4e43-8d91-107f2acaf7b5","Type":"ContainerStarted","Data":"1742a98a4380c89cae9ffd1302a0525d69c2620bab849566087da7859feab32e"} Jul 06 23:49:17.649227 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:17.649199 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7gntn" event={"ID":"e7b30a60-5ac4-43d8-891c-d23dd770b332","Type":"ContainerStarted","Data":"6190d4c664df69eb3532dcdd4812620b00fdbf6203ae781f20d60b88c90fe08c"} Jul 06 23:49:17.671265 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:17.671226 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6r2vp" podStartSLOduration=8.794662426 podStartE2EDuration="39.671211963s" totalImagesPullingTime="30.876549537s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:17.670082504 +0000 UTC m=+39.879812026" watchObservedRunningTime="2026-07-06 23:49:17.671211963 +0000 UTC m=+39.880941462" Jul 06 23:49:17.688351 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:17.688289 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7gntn" podStartSLOduration=1.338626126 podStartE2EDuration="4.688278602s" totalImagesPullingTime="3.349652476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:17.687791384 +0000 UTC m=+39.897520910" watchObservedRunningTime="2026-07-06 23:49:17.688278602 +0000 UTC m=+39.898008122" Jul 06 23:49:21.001876 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:21.001831 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/05a76f14-b4fd-4087-bd7a-53587b38ea81-metrics-tls\") pod \"dns-default-v978g\" (UID: \"05a76f14-b4fd-4087-bd7a-53587b38ea81\") " pod="openshift-dns/dns-default-v978g" Jul 06 23:49:21.001876 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:21.001883 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-tls\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:21.002312 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:21.001943 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 23:49:21.002312 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:21.001972 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 23:49:21.002312 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:21.001983 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c586d98b6-vr5mc: secret "image-registry-tls" not found Jul 06 23:49:21.002312 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:21.002006 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05a76f14-b4fd-4087-bd7a-53587b38ea81-metrics-tls podName:05a76f14-b4fd-4087-bd7a-53587b38ea81 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:29.001990873 +0000 UTC m=+51.211720373 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/05a76f14-b4fd-4087-bd7a-53587b38ea81-metrics-tls") pod "dns-default-v978g" (UID: "05a76f14-b4fd-4087-bd7a-53587b38ea81") : secret "dns-default-metrics-tls" not found Jul 06 23:49:21.002312 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:21.002021 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-tls podName:1bc2dc5e-4eab-40a3-9720-ff2ae2746cad nodeName:}" failed. No retries permitted until 2026-07-06 23:49:29.002014671 +0000 UTC m=+51.211744170 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-tls") pod "image-registry-7c586d98b6-vr5mc" (UID: "1bc2dc5e-4eab-40a3-9720-ff2ae2746cad") : secret "image-registry-tls" not found Jul 06 23:49:29.057819 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:29.057771 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/05a76f14-b4fd-4087-bd7a-53587b38ea81-metrics-tls\") pod \"dns-default-v978g\" (UID: \"05a76f14-b4fd-4087-bd7a-53587b38ea81\") " pod="openshift-dns/dns-default-v978g" Jul 06 23:49:29.057819 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:29.057818 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-tls\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:29.058420 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:29.057921 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 23:49:29.058420 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:29.057932 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c586d98b6-vr5mc: secret "image-registry-tls" not found Jul 06 23:49:29.058420 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:29.057933 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 23:49:29.058420 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:29.057994 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-tls podName:1bc2dc5e-4eab-40a3-9720-ff2ae2746cad nodeName:}" failed. No retries permitted until 2026-07-06 23:49:45.057980838 +0000 UTC m=+67.267710337 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-tls") pod "image-registry-7c586d98b6-vr5mc" (UID: "1bc2dc5e-4eab-40a3-9720-ff2ae2746cad") : secret "image-registry-tls" not found Jul 06 23:49:29.058420 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:29.058007 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05a76f14-b4fd-4087-bd7a-53587b38ea81-metrics-tls podName:05a76f14-b4fd-4087-bd7a-53587b38ea81 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:45.058001143 +0000 UTC m=+67.267730643 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/05a76f14-b4fd-4087-bd7a-53587b38ea81-metrics-tls") pod "dns-default-v978g" (UID: "05a76f14-b4fd-4087-bd7a-53587b38ea81") : secret "dns-default-metrics-tls" not found Jul 06 23:49:30.863034 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:30.862997 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5m4nb"] Jul 06 23:49:30.878016 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:30.877991 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5m4nb" Jul 06 23:49:30.883251 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:30.883143 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 06 23:49:30.886544 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:30.885979 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 23:49:30.886544 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:30.886434 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xk7b2\"" Jul 06 23:49:30.887709 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:30.887683 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 23:49:30.890071 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:30.888467 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 06 23:49:30.890184 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:30.889646 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5m4nb"] Jul 06 23:49:30.944620 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:30.944593 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nks6v"] Jul 06 23:49:30.966055 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:30.966023 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nks6v"] Jul 06 23:49:30.966219 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:30.966120 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nks6v" Jul 06 23:49:30.969304 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:30.969265 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 06 23:49:30.969728 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:30.969709 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 06 23:49:30.970057 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:30.970038 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 06 23:49:30.970149 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:30.970077 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-94b57\"" Jul 06 23:49:30.972738 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:30.972721 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/450852fa-55ab-4354-8781-e67dd3330970-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-5m4nb\" (UID: \"450852fa-55ab-4354-8781-e67dd3330970\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5m4nb" Jul 06 23:49:30.972810 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:30.972749 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/450852fa-55ab-4354-8781-e67dd3330970-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-5m4nb\" (UID: \"450852fa-55ab-4354-8781-e67dd3330970\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5m4nb" Jul 06 23:49:30.972865 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:30.972851 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7db6x\" (UniqueName: \"kubernetes.io/projected/450852fa-55ab-4354-8781-e67dd3330970-kube-api-access-7db6x\") pod \"cluster-monitoring-operator-64786d9cbb-5m4nb\" (UID: \"450852fa-55ab-4354-8781-e67dd3330970\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5m4nb" Jul 06 23:49:30.975117 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:30.975101 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 06 23:49:31.073979 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:31.073940 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/450852fa-55ab-4354-8781-e67dd3330970-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-5m4nb\" (UID: \"450852fa-55ab-4354-8781-e67dd3330970\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5m4nb" Jul 06 23:49:31.073979 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:31.073979 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/450852fa-55ab-4354-8781-e67dd3330970-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-5m4nb\" (UID: \"450852fa-55ab-4354-8781-e67dd3330970\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5m4nb" Jul 06 23:49:31.074199 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:31.074050 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7db6x\" (UniqueName: \"kubernetes.io/projected/450852fa-55ab-4354-8781-e67dd3330970-kube-api-access-7db6x\") pod \"cluster-monitoring-operator-64786d9cbb-5m4nb\" (UID: \"450852fa-55ab-4354-8781-e67dd3330970\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5m4nb" Jul 06 23:49:31.074199 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:31.074072 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5e219958-dd12-412f-920e-d732d6c4f806-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-nks6v\" (UID: \"5e219958-dd12-412f-920e-d732d6c4f806\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nks6v" Jul 06 23:49:31.074199 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:31.074095 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5e219958-dd12-412f-920e-d732d6c4f806-config\") pod \"service-ca-operator-5d46d9ffd4-nks6v\" (UID: \"5e219958-dd12-412f-920e-d732d6c4f806\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nks6v" Jul 06 23:49:31.074199 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:31.074141 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9ccp\" (UniqueName: \"kubernetes.io/projected/5e219958-dd12-412f-920e-d732d6c4f806-kube-api-access-d9ccp\") pod \"service-ca-operator-5d46d9ffd4-nks6v\" (UID: \"5e219958-dd12-412f-920e-d732d6c4f806\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nks6v" Jul 06 23:49:31.074199 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:31.074174 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 23:49:31.074385 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:31.074227 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/450852fa-55ab-4354-8781-e67dd3330970-cluster-monitoring-operator-tls podName:450852fa-55ab-4354-8781-e67dd3330970 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:31.574213204 +0000 UTC m=+53.783942703 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/450852fa-55ab-4354-8781-e67dd3330970-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-5m4nb" (UID: "450852fa-55ab-4354-8781-e67dd3330970") : secret "cluster-monitoring-operator-tls" not found Jul 06 23:49:31.074732 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:31.074713 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/450852fa-55ab-4354-8781-e67dd3330970-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-5m4nb\" (UID: \"450852fa-55ab-4354-8781-e67dd3330970\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5m4nb" Jul 06 23:49:31.084227 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:31.084207 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7db6x\" (UniqueName: \"kubernetes.io/projected/450852fa-55ab-4354-8781-e67dd3330970-kube-api-access-7db6x\") pod \"cluster-monitoring-operator-64786d9cbb-5m4nb\" (UID: \"450852fa-55ab-4354-8781-e67dd3330970\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5m4nb" Jul 06 23:49:31.175030 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:31.175000 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5e219958-dd12-412f-920e-d732d6c4f806-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-nks6v\" (UID: \"5e219958-dd12-412f-920e-d732d6c4f806\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nks6v" Jul 06 23:49:31.175030 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:31.175033 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5e219958-dd12-412f-920e-d732d6c4f806-config\") pod \"service-ca-operator-5d46d9ffd4-nks6v\" (UID: \"5e219958-dd12-412f-920e-d732d6c4f806\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nks6v" Jul 06 23:49:31.175276 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:31.175065 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9ccp\" (UniqueName: \"kubernetes.io/projected/5e219958-dd12-412f-920e-d732d6c4f806-kube-api-access-d9ccp\") pod \"service-ca-operator-5d46d9ffd4-nks6v\" (UID: \"5e219958-dd12-412f-920e-d732d6c4f806\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nks6v" Jul 06 23:49:31.175582 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:31.175563 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5e219958-dd12-412f-920e-d732d6c4f806-config\") pod \"service-ca-operator-5d46d9ffd4-nks6v\" (UID: \"5e219958-dd12-412f-920e-d732d6c4f806\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nks6v" Jul 06 23:49:31.177227 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:31.177204 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5e219958-dd12-412f-920e-d732d6c4f806-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-nks6v\" (UID: \"5e219958-dd12-412f-920e-d732d6c4f806\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nks6v" Jul 06 23:49:31.183442 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:31.183421 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9ccp\" (UniqueName: \"kubernetes.io/projected/5e219958-dd12-412f-920e-d732d6c4f806-kube-api-access-d9ccp\") pod \"service-ca-operator-5d46d9ffd4-nks6v\" (UID: \"5e219958-dd12-412f-920e-d732d6c4f806\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nks6v" Jul 06 23:49:31.274676 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:31.274637 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nks6v" Jul 06 23:49:31.413457 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:31.413424 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nks6v"] Jul 06 23:49:31.416900 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:49:31.416872 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e219958_dd12_412f_920e_d732d6c4f806.slice/crio-9a60dfe8216e914a0017260d3a7fafdc431d02ec554edb733420fa27a175ccad WatchSource:0}: Error finding container 9a60dfe8216e914a0017260d3a7fafdc431d02ec554edb733420fa27a175ccad: Status 404 returned error can't find the container with id 9a60dfe8216e914a0017260d3a7fafdc431d02ec554edb733420fa27a175ccad Jul 06 23:49:31.578752 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:31.578645 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/450852fa-55ab-4354-8781-e67dd3330970-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-5m4nb\" (UID: \"450852fa-55ab-4354-8781-e67dd3330970\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5m4nb" Jul 06 23:49:31.578906 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:31.578811 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 23:49:31.578906 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:31.578886 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/450852fa-55ab-4354-8781-e67dd3330970-cluster-monitoring-operator-tls podName:450852fa-55ab-4354-8781-e67dd3330970 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:32.578869893 +0000 UTC m=+54.788599392 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/450852fa-55ab-4354-8781-e67dd3330970-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-5m4nb" (UID: "450852fa-55ab-4354-8781-e67dd3330970") : secret "cluster-monitoring-operator-tls" not found Jul 06 23:49:31.680255 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:31.680214 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nks6v" event={"ID":"5e219958-dd12-412f-920e-d732d6c4f806","Type":"ContainerStarted","Data":"9a60dfe8216e914a0017260d3a7fafdc431d02ec554edb733420fa27a175ccad"} Jul 06 23:49:32.586758 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:32.586717 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/450852fa-55ab-4354-8781-e67dd3330970-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-5m4nb\" (UID: \"450852fa-55ab-4354-8781-e67dd3330970\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5m4nb" Jul 06 23:49:32.587144 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:32.586877 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 23:49:32.587144 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:32.586947 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/450852fa-55ab-4354-8781-e67dd3330970-cluster-monitoring-operator-tls podName:450852fa-55ab-4354-8781-e67dd3330970 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:34.586930811 +0000 UTC m=+56.796660311 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/450852fa-55ab-4354-8781-e67dd3330970-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-5m4nb" (UID: "450852fa-55ab-4354-8781-e67dd3330970") : secret "cluster-monitoring-operator-tls" not found Jul 06 23:49:33.686746 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:33.686710 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nks6v" event={"ID":"5e219958-dd12-412f-920e-d732d6c4f806","Type":"ContainerStarted","Data":"6043fd5b7733b8dd9b0301f409b5b1c78ec80b4e0c158f7abe6853c41ebe259b"} Jul 06 23:49:33.703368 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:33.703314 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nks6v" podStartSLOduration=1.5987981530000002 podStartE2EDuration="3.703282207s" totalImagesPullingTime="2.104484054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:33.702592946 +0000 UTC m=+55.912322487" watchObservedRunningTime="2026-07-06 23:49:33.703282207 +0000 UTC m=+55.913011729" Jul 06 23:49:34.604613 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:34.604555 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/450852fa-55ab-4354-8781-e67dd3330970-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-5m4nb\" (UID: \"450852fa-55ab-4354-8781-e67dd3330970\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5m4nb" Jul 06 23:49:34.604899 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:34.604708 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 23:49:34.604899 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:34.604776 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/450852fa-55ab-4354-8781-e67dd3330970-cluster-monitoring-operator-tls podName:450852fa-55ab-4354-8781-e67dd3330970 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:38.604761156 +0000 UTC m=+60.814490654 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/450852fa-55ab-4354-8781-e67dd3330970-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-5m4nb" (UID: "450852fa-55ab-4354-8781-e67dd3330970") : secret "cluster-monitoring-operator-tls" not found Jul 06 23:49:37.979117 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:37.979084 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rt6cm"] Jul 06 23:49:38.010429 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:38.010392 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rt6cm"] Jul 06 23:49:38.010564 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:38.010522 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rt6cm" Jul 06 23:49:38.013163 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:38.013143 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 23:49:38.014470 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:38.014453 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gbs9b\"" Jul 06 23:49:38.014586 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:38.014487 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 23:49:38.014586 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:38.014505 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 23:49:38.014586 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:38.014486 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 23:49:38.136453 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:38.136420 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/44cf08e2-0939-4411-8f46-7407b0e66e33-data-volume\") pod \"insights-runtime-extractor-rt6cm\" (UID: \"44cf08e2-0939-4411-8f46-7407b0e66e33\") " pod="openshift-insights/insights-runtime-extractor-rt6cm" Jul 06 23:49:38.136617 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:38.136457 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/44cf08e2-0939-4411-8f46-7407b0e66e33-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rt6cm\" (UID: \"44cf08e2-0939-4411-8f46-7407b0e66e33\") " pod="openshift-insights/insights-runtime-extractor-rt6cm" Jul 06 23:49:38.136617 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:38.136489 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t6hf\" (UniqueName: \"kubernetes.io/projected/44cf08e2-0939-4411-8f46-7407b0e66e33-kube-api-access-7t6hf\") pod \"insights-runtime-extractor-rt6cm\" (UID: \"44cf08e2-0939-4411-8f46-7407b0e66e33\") " pod="openshift-insights/insights-runtime-extractor-rt6cm" Jul 06 23:49:38.136692 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:38.136622 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/44cf08e2-0939-4411-8f46-7407b0e66e33-crio-socket\") pod \"insights-runtime-extractor-rt6cm\" (UID: \"44cf08e2-0939-4411-8f46-7407b0e66e33\") " pod="openshift-insights/insights-runtime-extractor-rt6cm" Jul 06 23:49:38.136692 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:38.136655 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/44cf08e2-0939-4411-8f46-7407b0e66e33-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rt6cm\" (UID: \"44cf08e2-0939-4411-8f46-7407b0e66e33\") " pod="openshift-insights/insights-runtime-extractor-rt6cm" Jul 06 23:49:38.237318 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:38.237205 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/44cf08e2-0939-4411-8f46-7407b0e66e33-crio-socket\") pod \"insights-runtime-extractor-rt6cm\" (UID: \"44cf08e2-0939-4411-8f46-7407b0e66e33\") " pod="openshift-insights/insights-runtime-extractor-rt6cm" Jul 06 23:49:38.237318 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:38.237248 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/44cf08e2-0939-4411-8f46-7407b0e66e33-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rt6cm\" (UID: \"44cf08e2-0939-4411-8f46-7407b0e66e33\") " pod="openshift-insights/insights-runtime-extractor-rt6cm" Jul 06 23:49:38.237318 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:38.237275 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/44cf08e2-0939-4411-8f46-7407b0e66e33-data-volume\") pod \"insights-runtime-extractor-rt6cm\" (UID: \"44cf08e2-0939-4411-8f46-7407b0e66e33\") " pod="openshift-insights/insights-runtime-extractor-rt6cm" Jul 06 23:49:38.237588 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:38.237360 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/44cf08e2-0939-4411-8f46-7407b0e66e33-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rt6cm\" (UID: \"44cf08e2-0939-4411-8f46-7407b0e66e33\") " pod="openshift-insights/insights-runtime-extractor-rt6cm" Jul 06 23:49:38.237588 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:38.237411 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t6hf\" (UniqueName: \"kubernetes.io/projected/44cf08e2-0939-4411-8f46-7407b0e66e33-kube-api-access-7t6hf\") pod \"insights-runtime-extractor-rt6cm\" (UID: \"44cf08e2-0939-4411-8f46-7407b0e66e33\") " pod="openshift-insights/insights-runtime-extractor-rt6cm" Jul 06 23:49:38.237588 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:38.237365 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/44cf08e2-0939-4411-8f46-7407b0e66e33-crio-socket\") pod \"insights-runtime-extractor-rt6cm\" (UID: \"44cf08e2-0939-4411-8f46-7407b0e66e33\") " pod="openshift-insights/insights-runtime-extractor-rt6cm" Jul 06 23:49:38.237681 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:38.237627 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/44cf08e2-0939-4411-8f46-7407b0e66e33-data-volume\") pod \"insights-runtime-extractor-rt6cm\" (UID: \"44cf08e2-0939-4411-8f46-7407b0e66e33\") " pod="openshift-insights/insights-runtime-extractor-rt6cm" Jul 06 23:49:38.239782 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:38.239759 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 23:49:38.239782 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:38.239770 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 23:49:38.245749 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:38.245730 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 23:49:38.247767 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:38.247753 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 23:49:38.247824 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:38.247778 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 23:49:38.247824 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:38.247815 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/44cf08e2-0939-4411-8f46-7407b0e66e33-kube-rbac-proxy-cm podName:44cf08e2-0939-4411-8f46-7407b0e66e33 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:38.74780018 +0000 UTC m=+60.957529682 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/44cf08e2-0939-4411-8f46-7407b0e66e33-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rt6cm" (UID: "44cf08e2-0939-4411-8f46-7407b0e66e33") : configmap "kube-rbac-proxy" not found Jul 06 23:49:38.247897 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:38.247833 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44cf08e2-0939-4411-8f46-7407b0e66e33-insights-runtime-extractor-tls podName:44cf08e2-0939-4411-8f46-7407b0e66e33 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:38.747823289 +0000 UTC m=+60.957552788 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/44cf08e2-0939-4411-8f46-7407b0e66e33-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rt6cm" (UID: "44cf08e2-0939-4411-8f46-7407b0e66e33") : secret "insights-runtime-extractor-tls" not found Jul 06 23:49:38.256733 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:38.256710 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 23:49:38.266681 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:38.266653 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t6hf\" (UniqueName: \"kubernetes.io/projected/44cf08e2-0939-4411-8f46-7407b0e66e33-kube-api-access-7t6hf\") pod \"insights-runtime-extractor-rt6cm\" (UID: \"44cf08e2-0939-4411-8f46-7407b0e66e33\") " pod="openshift-insights/insights-runtime-extractor-rt6cm" Jul 06 23:49:38.640840 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:38.640738 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/450852fa-55ab-4354-8781-e67dd3330970-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-5m4nb\" (UID: \"450852fa-55ab-4354-8781-e67dd3330970\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5m4nb" Jul 06 23:49:38.640997 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:38.640897 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 23:49:38.640997 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:38.640969 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/450852fa-55ab-4354-8781-e67dd3330970-cluster-monitoring-operator-tls podName:450852fa-55ab-4354-8781-e67dd3330970 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:46.640954494 +0000 UTC m=+68.850683992 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/450852fa-55ab-4354-8781-e67dd3330970-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-5m4nb" (UID: "450852fa-55ab-4354-8781-e67dd3330970") : secret "cluster-monitoring-operator-tls" not found Jul 06 23:49:38.842732 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:38.842682 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/44cf08e2-0939-4411-8f46-7407b0e66e33-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rt6cm\" (UID: \"44cf08e2-0939-4411-8f46-7407b0e66e33\") " pod="openshift-insights/insights-runtime-extractor-rt6cm" Jul 06 23:49:38.842938 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:38.842747 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/44cf08e2-0939-4411-8f46-7407b0e66e33-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rt6cm\" (UID: \"44cf08e2-0939-4411-8f46-7407b0e66e33\") " pod="openshift-insights/insights-runtime-extractor-rt6cm" Jul 06 23:49:38.842938 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:38.842810 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 23:49:38.842938 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:38.842893 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/44cf08e2-0939-4411-8f46-7407b0e66e33-kube-rbac-proxy-cm podName:44cf08e2-0939-4411-8f46-7407b0e66e33 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:39.842877232 +0000 UTC m=+62.052606731 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/44cf08e2-0939-4411-8f46-7407b0e66e33-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rt6cm" (UID: "44cf08e2-0939-4411-8f46-7407b0e66e33") : configmap "kube-rbac-proxy" not found Jul 06 23:49:38.842938 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:38.842894 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 23:49:38.843165 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:38.842951 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44cf08e2-0939-4411-8f46-7407b0e66e33-insights-runtime-extractor-tls podName:44cf08e2-0939-4411-8f46-7407b0e66e33 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:39.842934058 +0000 UTC m=+62.052663557 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/44cf08e2-0939-4411-8f46-7407b0e66e33-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rt6cm" (UID: "44cf08e2-0939-4411-8f46-7407b0e66e33") : secret "insights-runtime-extractor-tls" not found Jul 06 23:49:39.619383 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:39.619358 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4z8vq" Jul 06 23:49:39.852061 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:39.852023 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/44cf08e2-0939-4411-8f46-7407b0e66e33-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rt6cm\" (UID: \"44cf08e2-0939-4411-8f46-7407b0e66e33\") " pod="openshift-insights/insights-runtime-extractor-rt6cm" Jul 06 23:49:39.852234 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:39.852172 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/44cf08e2-0939-4411-8f46-7407b0e66e33-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rt6cm\" (UID: \"44cf08e2-0939-4411-8f46-7407b0e66e33\") " pod="openshift-insights/insights-runtime-extractor-rt6cm" Jul 06 23:49:39.852234 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:39.852174 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 23:49:39.852234 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:39.852215 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 23:49:39.852368 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:39.852272 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/44cf08e2-0939-4411-8f46-7407b0e66e33-kube-rbac-proxy-cm podName:44cf08e2-0939-4411-8f46-7407b0e66e33 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:41.852258666 +0000 UTC m=+64.061988180 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/44cf08e2-0939-4411-8f46-7407b0e66e33-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rt6cm" (UID: "44cf08e2-0939-4411-8f46-7407b0e66e33") : configmap "kube-rbac-proxy" not found Jul 06 23:49:39.852368 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:39.852287 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44cf08e2-0939-4411-8f46-7407b0e66e33-insights-runtime-extractor-tls podName:44cf08e2-0939-4411-8f46-7407b0e66e33 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:41.852280527 +0000 UTC m=+64.062010025 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/44cf08e2-0939-4411-8f46-7407b0e66e33-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rt6cm" (UID: "44cf08e2-0939-4411-8f46-7407b0e66e33") : secret "insights-runtime-extractor-tls" not found Jul 06 23:49:40.554947 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:40.554917 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9tms8_93308370-a961-4b85-a764-abac1d17c520/dns-node-resolver/0.log" Jul 06 23:49:41.763694 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:41.763664 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tdk4b_4f2c8799-a48e-4f4c-bce4-7f0187c24298/node-ca/0.log" Jul 06 23:49:41.865260 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:41.865220 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/44cf08e2-0939-4411-8f46-7407b0e66e33-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rt6cm\" (UID: \"44cf08e2-0939-4411-8f46-7407b0e66e33\") " pod="openshift-insights/insights-runtime-extractor-rt6cm" Jul 06 23:49:41.865449 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:41.865394 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 23:49:41.865502 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:41.865467 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/44cf08e2-0939-4411-8f46-7407b0e66e33-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rt6cm\" (UID: \"44cf08e2-0939-4411-8f46-7407b0e66e33\") " pod="openshift-insights/insights-runtime-extractor-rt6cm" Jul 06 23:49:41.865502 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:41.865483 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44cf08e2-0939-4411-8f46-7407b0e66e33-insights-runtime-extractor-tls podName:44cf08e2-0939-4411-8f46-7407b0e66e33 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:45.865462315 +0000 UTC m=+68.075191815 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/44cf08e2-0939-4411-8f46-7407b0e66e33-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rt6cm" (UID: "44cf08e2-0939-4411-8f46-7407b0e66e33") : secret "insights-runtime-extractor-tls" not found Jul 06 23:49:41.865583 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:41.865522 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 23:49:41.865583 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:41.865574 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/44cf08e2-0939-4411-8f46-7407b0e66e33-kube-rbac-proxy-cm podName:44cf08e2-0939-4411-8f46-7407b0e66e33 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:45.865561652 +0000 UTC m=+68.075291163 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/44cf08e2-0939-4411-8f46-7407b0e66e33-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rt6cm" (UID: "44cf08e2-0939-4411-8f46-7407b0e66e33") : configmap "kube-rbac-proxy" not found Jul 06 23:49:42.161219 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:42.161188 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7gntn_e7b30a60-5ac4-43d8-891c-d23dd770b332/serve-healthcheck-canary/0.log" Jul 06 23:49:43.981992 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:43.981944 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68c477a5-72a1-47ec-9864-17cd0021a5f9-original-pull-secret\") pod \"global-pull-secret-syncer-55rgj\" (UID: \"68c477a5-72a1-47ec-9864-17cd0021a5f9\") " pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:49:43.984789 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:43.984770 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 23:49:43.995405 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:43.995378 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68c477a5-72a1-47ec-9864-17cd0021a5f9-original-pull-secret\") pod \"global-pull-secret-syncer-55rgj\" (UID: \"68c477a5-72a1-47ec-9864-17cd0021a5f9\") " pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:49:44.082623 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:44.082581 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/971187a6-5709-48db-b990-0a13db9fad13-metrics-certs\") pod \"network-metrics-daemon-zmkd8\" (UID: \"971187a6-5709-48db-b990-0a13db9fad13\") " pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:49:44.082820 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:44.082640 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp29w\" (UniqueName: \"kubernetes.io/projected/8f45c29d-2b22-4a11-bdc7-2e8dc27aab32-kube-api-access-sp29w\") pod \"network-check-target-qndcn\" (UID: \"8f45c29d-2b22-4a11-bdc7-2e8dc27aab32\") " pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:49:44.085625 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:44.085606 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 23:49:44.086140 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:44.086124 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 23:49:44.093261 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:44.093239 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 06 23:49:44.093334 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:44.093320 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/971187a6-5709-48db-b990-0a13db9fad13-metrics-certs podName:971187a6-5709-48db-b990-0a13db9fad13 nodeName:}" failed. No retries permitted until 2026-07-06 23:50:48.093284081 +0000 UTC m=+130.303013584 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/971187a6-5709-48db-b990-0a13db9fad13-metrics-certs") pod "network-metrics-daemon-zmkd8" (UID: "971187a6-5709-48db-b990-0a13db9fad13") : secret "metrics-daemon-secret" not found Jul 06 23:49:44.096617 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:44.096601 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 23:49:44.106033 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:44.106004 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sp29w\" (UniqueName: \"kubernetes.io/projected/8f45c29d-2b22-4a11-bdc7-2e8dc27aab32-kube-api-access-sp29w\") pod \"network-check-target-qndcn\" (UID: \"8f45c29d-2b22-4a11-bdc7-2e8dc27aab32\") " pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:49:44.211570 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:44.211537 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dwstx\"" Jul 06 23:49:44.214933 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:44.214917 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55rgj" Jul 06 23:49:44.219106 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:44.219090 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:49:44.349809 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:44.349774 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-55rgj"] Jul 06 23:49:44.353634 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:49:44.353608 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68c477a5_72a1_47ec_9864_17cd0021a5f9.slice/crio-a7fdfc0b088cc5682fca7f4a83a9b4efeb20ba63975e11b1e83197ef803d0776 WatchSource:0}: Error finding container a7fdfc0b088cc5682fca7f4a83a9b4efeb20ba63975e11b1e83197ef803d0776: Status 404 returned error can't find the container with id a7fdfc0b088cc5682fca7f4a83a9b4efeb20ba63975e11b1e83197ef803d0776 Jul 06 23:49:44.363326 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:44.363262 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qndcn"] Jul 06 23:49:44.367112 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:49:44.367088 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f45c29d_2b22_4a11_bdc7_2e8dc27aab32.slice/crio-d2ff56b0bacc692eb29b419b66dd69766796f5140e56ac8e23c6d8f1b4888db5 WatchSource:0}: Error finding container d2ff56b0bacc692eb29b419b66dd69766796f5140e56ac8e23c6d8f1b4888db5: Status 404 returned error can't find the container with id d2ff56b0bacc692eb29b419b66dd69766796f5140e56ac8e23c6d8f1b4888db5 Jul 06 23:49:44.715821 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:44.715785 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-55rgj" event={"ID":"68c477a5-72a1-47ec-9864-17cd0021a5f9","Type":"ContainerStarted","Data":"a7fdfc0b088cc5682fca7f4a83a9b4efeb20ba63975e11b1e83197ef803d0776"} Jul 06 23:49:44.716797 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:44.716774 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qndcn" event={"ID":"8f45c29d-2b22-4a11-bdc7-2e8dc27aab32","Type":"ContainerStarted","Data":"d2ff56b0bacc692eb29b419b66dd69766796f5140e56ac8e23c6d8f1b4888db5"} Jul 06 23:49:45.090890 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:45.090375 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/05a76f14-b4fd-4087-bd7a-53587b38ea81-metrics-tls\") pod \"dns-default-v978g\" (UID: \"05a76f14-b4fd-4087-bd7a-53587b38ea81\") " pod="openshift-dns/dns-default-v978g" Jul 06 23:49:45.090890 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:45.090417 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-tls\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:45.093507 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:45.093475 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/05a76f14-b4fd-4087-bd7a-53587b38ea81-metrics-tls\") pod \"dns-default-v978g\" (UID: \"05a76f14-b4fd-4087-bd7a-53587b38ea81\") " pod="openshift-dns/dns-default-v978g" Jul 06 23:49:45.093611 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:45.093537 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-tls\") pod \"image-registry-7c586d98b6-vr5mc\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:45.322658 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:45.322625 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-csfd5\"" Jul 06 23:49:45.330405 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:45.330379 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:45.348974 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:45.348908 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-k5jpj\"" Jul 06 23:49:45.357203 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:45.357011 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v978g" Jul 06 23:49:45.507462 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:45.507420 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c586d98b6-vr5mc"] Jul 06 23:49:45.511761 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:49:45.511725 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bc2dc5e_4eab_40a3_9720_ff2ae2746cad.slice/crio-0aa641940ea12e8d755118a4af3c1fd40a948188c22838de9dddbbad5589c581 WatchSource:0}: Error finding container 0aa641940ea12e8d755118a4af3c1fd40a948188c22838de9dddbbad5589c581: Status 404 returned error can't find the container with id 0aa641940ea12e8d755118a4af3c1fd40a948188c22838de9dddbbad5589c581 Jul 06 23:49:45.532731 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:45.532692 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-v978g"] Jul 06 23:49:45.538286 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:49:45.538256 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05a76f14_b4fd_4087_bd7a_53587b38ea81.slice/crio-cb0399880d0e94c2675f41a32194844b79c2de738e5a1df7ed7529a3eb5ad550 WatchSource:0}: Error finding container cb0399880d0e94c2675f41a32194844b79c2de738e5a1df7ed7529a3eb5ad550: Status 404 returned error can't find the container with id cb0399880d0e94c2675f41a32194844b79c2de738e5a1df7ed7529a3eb5ad550 Jul 06 23:49:45.721109 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:45.721070 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v978g" event={"ID":"05a76f14-b4fd-4087-bd7a-53587b38ea81","Type":"ContainerStarted","Data":"cb0399880d0e94c2675f41a32194844b79c2de738e5a1df7ed7529a3eb5ad550"} Jul 06 23:49:45.722925 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:45.722896 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" event={"ID":"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad","Type":"ContainerStarted","Data":"367d1741105f67257af66952de7e959bed07931970a9a56cf058e06f0d7b6802"} Jul 06 23:49:45.723048 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:45.722940 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" event={"ID":"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad","Type":"ContainerStarted","Data":"0aa641940ea12e8d755118a4af3c1fd40a948188c22838de9dddbbad5589c581"} Jul 06 23:49:45.723048 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:45.723035 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:49:45.746436 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:45.745919 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" podStartSLOduration=67.745891734 podStartE2EDuration="1m7.745891734s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:48:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:45.743875265 +0000 UTC m=+67.953604831" watchObservedRunningTime="2026-07-06 23:49:45.745891734 +0000 UTC m=+67.955621257" Jul 06 23:49:45.897414 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:45.897328 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/44cf08e2-0939-4411-8f46-7407b0e66e33-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rt6cm\" (UID: \"44cf08e2-0939-4411-8f46-7407b0e66e33\") " pod="openshift-insights/insights-runtime-extractor-rt6cm" Jul 06 23:49:45.897414 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:45.897401 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/44cf08e2-0939-4411-8f46-7407b0e66e33-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rt6cm\" (UID: \"44cf08e2-0939-4411-8f46-7407b0e66e33\") " pod="openshift-insights/insights-runtime-extractor-rt6cm" Jul 06 23:49:45.897604 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:45.897529 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 23:49:45.897604 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:45.897593 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44cf08e2-0939-4411-8f46-7407b0e66e33-insights-runtime-extractor-tls podName:44cf08e2-0939-4411-8f46-7407b0e66e33 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:53.897573255 +0000 UTC m=+76.107302775 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/44cf08e2-0939-4411-8f46-7407b0e66e33-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rt6cm" (UID: "44cf08e2-0939-4411-8f46-7407b0e66e33") : secret "insights-runtime-extractor-tls" not found Jul 06 23:49:45.898020 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:45.897998 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 23:49:45.898094 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:45.898065 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/44cf08e2-0939-4411-8f46-7407b0e66e33-kube-rbac-proxy-cm podName:44cf08e2-0939-4411-8f46-7407b0e66e33 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:53.898048543 +0000 UTC m=+76.107778048 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/44cf08e2-0939-4411-8f46-7407b0e66e33-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rt6cm" (UID: "44cf08e2-0939-4411-8f46-7407b0e66e33") : configmap "kube-rbac-proxy" not found Jul 06 23:49:46.705896 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:46.705852 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/450852fa-55ab-4354-8781-e67dd3330970-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-5m4nb\" (UID: \"450852fa-55ab-4354-8781-e67dd3330970\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5m4nb" Jul 06 23:49:46.706434 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:46.706035 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 23:49:46.706434 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:49:46.706132 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/450852fa-55ab-4354-8781-e67dd3330970-cluster-monitoring-operator-tls podName:450852fa-55ab-4354-8781-e67dd3330970 nodeName:}" failed. No retries permitted until 2026-07-06 23:50:02.706110925 +0000 UTC m=+84.915840436 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/450852fa-55ab-4354-8781-e67dd3330970-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-5m4nb" (UID: "450852fa-55ab-4354-8781-e67dd3330970") : secret "cluster-monitoring-operator-tls" not found Jul 06 23:49:49.739037 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:49.738085 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v978g" event={"ID":"05a76f14-b4fd-4087-bd7a-53587b38ea81","Type":"ContainerStarted","Data":"08354e545bd450d2edbad868d3a7160682f4ccd7514628dd1c9cc27e282f8d84"} Jul 06 23:49:49.739037 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:49.738185 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v978g" event={"ID":"05a76f14-b4fd-4087-bd7a-53587b38ea81","Type":"ContainerStarted","Data":"459ea6cd61d1c40790f4c2629d1bf54f5af2379e7ef74d3cc54c2f94adc792c6"} Jul 06 23:49:49.739037 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:49.738963 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-v978g" Jul 06 23:49:49.741157 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:49.741086 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qndcn" event={"ID":"8f45c29d-2b22-4a11-bdc7-2e8dc27aab32","Type":"ContainerStarted","Data":"1029b80ee365aa86519dd6b1148d1377b0d98ff91e60fced46981801463db08c"} Jul 06 23:49:49.741278 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:49.741178 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:49:49.742600 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:49.742577 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-55rgj" event={"ID":"68c477a5-72a1-47ec-9864-17cd0021a5f9","Type":"ContainerStarted","Data":"e4acaf6c9fb33db9791b537bc9644c0312ad06acf9994297186f8a7ade3ec492"} Jul 06 23:49:49.763047 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:49.762996 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-v978g" podStartSLOduration=32.792462313 podStartE2EDuration="36.762981849s" totalImagesPullingTime="3.970519536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:49.762384181 +0000 UTC m=+71.972113723" watchObservedRunningTime="2026-07-06 23:49:49.762981849 +0000 UTC m=+71.972711369" Jul 06 23:49:49.780010 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:49.779952 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qndcn" podStartSLOduration=66.639701779 podStartE2EDuration="1m11.779934564s" totalImagesPullingTime="5.140232785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:49.779617003 +0000 UTC m=+71.989346525" watchObservedRunningTime="2026-07-06 23:49:49.779934564 +0000 UTC m=+71.989664086" Jul 06 23:49:53.961782 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:53.961728 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/44cf08e2-0939-4411-8f46-7407b0e66e33-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rt6cm\" (UID: \"44cf08e2-0939-4411-8f46-7407b0e66e33\") " pod="openshift-insights/insights-runtime-extractor-rt6cm" Jul 06 23:49:53.962411 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:53.961826 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/44cf08e2-0939-4411-8f46-7407b0e66e33-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rt6cm\" (UID: \"44cf08e2-0939-4411-8f46-7407b0e66e33\") " pod="openshift-insights/insights-runtime-extractor-rt6cm" Jul 06 23:49:53.962411 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:53.962283 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/44cf08e2-0939-4411-8f46-7407b0e66e33-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rt6cm\" (UID: \"44cf08e2-0939-4411-8f46-7407b0e66e33\") " pod="openshift-insights/insights-runtime-extractor-rt6cm" Jul 06 23:49:53.964163 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:53.964135 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/44cf08e2-0939-4411-8f46-7407b0e66e33-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rt6cm\" (UID: \"44cf08e2-0939-4411-8f46-7407b0e66e33\") " pod="openshift-insights/insights-runtime-extractor-rt6cm" Jul 06 23:49:54.222909 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:54.222821 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gbs9b\"" Jul 06 23:49:54.230962 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:54.230935 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rt6cm" Jul 06 23:49:54.357654 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:54.357598 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-55rgj" podStartSLOduration=71.191588073 podStartE2EDuration="1m16.357581779s" totalImagesPullingTime="5.165993706s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:49.802177765 +0000 UTC m=+72.011907287" watchObservedRunningTime="2026-07-06 23:49:54.357581779 +0000 UTC m=+76.567311309" Jul 06 23:49:54.358365 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:54.358345 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rt6cm"] Jul 06 23:49:54.361367 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:49:54.361331 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44cf08e2_0939_4411_8f46_7407b0e66e33.slice/crio-22fe83271f08b137d2199861c8c246405003472ed7dc8d15b8731c99ab7cd1ca WatchSource:0}: Error finding container 22fe83271f08b137d2199861c8c246405003472ed7dc8d15b8731c99ab7cd1ca: Status 404 returned error can't find the container with id 22fe83271f08b137d2199861c8c246405003472ed7dc8d15b8731c99ab7cd1ca Jul 06 23:49:54.761312 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:54.761217 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rt6cm" event={"ID":"44cf08e2-0939-4411-8f46-7407b0e66e33","Type":"ContainerStarted","Data":"22fe83271f08b137d2199861c8c246405003472ed7dc8d15b8731c99ab7cd1ca"} Jul 06 23:49:57.843271 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:57.843232 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffcc8559-9vmvx"] Jul 06 23:49:57.846615 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:57.846595 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffcc8559-9vmvx" Jul 06 23:49:57.849920 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:57.849868 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 06 23:49:57.850525 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:57.850499 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 06 23:49:57.850944 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:57.850927 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 06 23:49:57.851842 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:57.851825 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 06 23:49:57.860325 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:57.860280 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffcc8559-9vmvx"] Jul 06 23:49:58.000814 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:58.000790 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/19494d5f-77e6-449a-b651-d104648d5dd9-klusterlet-config\") pod \"klusterlet-addon-workmgr-85ffcc8559-9vmvx\" (UID: \"19494d5f-77e6-449a-b651-d104648d5dd9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffcc8559-9vmvx" Jul 06 23:49:58.000926 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:58.000873 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl7nl\" (UniqueName: \"kubernetes.io/projected/19494d5f-77e6-449a-b651-d104648d5dd9-kube-api-access-nl7nl\") pod \"klusterlet-addon-workmgr-85ffcc8559-9vmvx\" (UID: \"19494d5f-77e6-449a-b651-d104648d5dd9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffcc8559-9vmvx" Jul 06 23:49:58.000926 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:58.000899 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19494d5f-77e6-449a-b651-d104648d5dd9-tmp\") pod \"klusterlet-addon-workmgr-85ffcc8559-9vmvx\" (UID: \"19494d5f-77e6-449a-b651-d104648d5dd9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffcc8559-9vmvx" Jul 06 23:49:58.102287 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:58.102211 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl7nl\" (UniqueName: \"kubernetes.io/projected/19494d5f-77e6-449a-b651-d104648d5dd9-kube-api-access-nl7nl\") pod \"klusterlet-addon-workmgr-85ffcc8559-9vmvx\" (UID: \"19494d5f-77e6-449a-b651-d104648d5dd9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffcc8559-9vmvx" Jul 06 23:49:58.102287 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:58.102243 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19494d5f-77e6-449a-b651-d104648d5dd9-tmp\") pod \"klusterlet-addon-workmgr-85ffcc8559-9vmvx\" (UID: \"19494d5f-77e6-449a-b651-d104648d5dd9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffcc8559-9vmvx" Jul 06 23:49:58.102526 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:58.102331 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/19494d5f-77e6-449a-b651-d104648d5dd9-klusterlet-config\") pod \"klusterlet-addon-workmgr-85ffcc8559-9vmvx\" (UID: \"19494d5f-77e6-449a-b651-d104648d5dd9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffcc8559-9vmvx" Jul 06 23:49:58.102717 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:58.102690 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19494d5f-77e6-449a-b651-d104648d5dd9-tmp\") pod \"klusterlet-addon-workmgr-85ffcc8559-9vmvx\" (UID: \"19494d5f-77e6-449a-b651-d104648d5dd9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffcc8559-9vmvx" Jul 06 23:49:58.104762 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:58.104741 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/19494d5f-77e6-449a-b651-d104648d5dd9-klusterlet-config\") pod \"klusterlet-addon-workmgr-85ffcc8559-9vmvx\" (UID: \"19494d5f-77e6-449a-b651-d104648d5dd9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffcc8559-9vmvx" Jul 06 23:49:58.111975 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:58.111952 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl7nl\" (UniqueName: \"kubernetes.io/projected/19494d5f-77e6-449a-b651-d104648d5dd9-kube-api-access-nl7nl\") pod \"klusterlet-addon-workmgr-85ffcc8559-9vmvx\" (UID: \"19494d5f-77e6-449a-b651-d104648d5dd9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffcc8559-9vmvx" Jul 06 23:49:58.158688 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:58.158643 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffcc8559-9vmvx" Jul 06 23:49:58.287390 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:58.287361 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffcc8559-9vmvx"] Jul 06 23:49:58.291091 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:49:58.291064 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19494d5f_77e6_449a_b651_d104648d5dd9.slice/crio-eece7e8801fe27d75e0ccaeb19ce0567596724d49f941ac64d4f78025a06a00a WatchSource:0}: Error finding container eece7e8801fe27d75e0ccaeb19ce0567596724d49f941ac64d4f78025a06a00a: Status 404 returned error can't find the container with id eece7e8801fe27d75e0ccaeb19ce0567596724d49f941ac64d4f78025a06a00a Jul 06 23:49:58.775288 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:58.775251 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rt6cm" event={"ID":"44cf08e2-0939-4411-8f46-7407b0e66e33","Type":"ContainerStarted","Data":"0cb6e2a50c9d0c85558c8debe55fc3009c59159debab40b815a4a300746c3d1d"} Jul 06 23:49:58.775483 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:58.775369 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rt6cm" event={"ID":"44cf08e2-0939-4411-8f46-7407b0e66e33","Type":"ContainerStarted","Data":"92999c81aa86af6651625fd3ffefcf807dfe0b2e9df364f1b28ce80fafa420da"} Jul 06 23:49:58.776172 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:58.776148 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffcc8559-9vmvx" event={"ID":"19494d5f-77e6-449a-b651-d104648d5dd9","Type":"ContainerStarted","Data":"eece7e8801fe27d75e0ccaeb19ce0567596724d49f941ac64d4f78025a06a00a"} Jul 06 23:49:59.546967 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:59.546928 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-bvlqn"] Jul 06 23:49:59.551320 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:59.550368 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-bvlqn" Jul 06 23:49:59.554622 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:59.554123 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-tk5lr\"" Jul 06 23:49:59.554622 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:59.554138 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 23:49:59.554622 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:59.554427 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 23:49:59.562219 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:59.562055 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-bvlqn"] Jul 06 23:49:59.615845 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:59.615807 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhrkc\" (UniqueName: \"kubernetes.io/projected/a5c9b8dc-3d47-439f-89bf-01f9077f4043-kube-api-access-zhrkc\") pod \"downloads-685bfb5f64-bvlqn\" (UID: \"a5c9b8dc-3d47-439f-89bf-01f9077f4043\") " pod="openshift-console/downloads-685bfb5f64-bvlqn" Jul 06 23:49:59.716634 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:59.716595 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhrkc\" (UniqueName: \"kubernetes.io/projected/a5c9b8dc-3d47-439f-89bf-01f9077f4043-kube-api-access-zhrkc\") pod \"downloads-685bfb5f64-bvlqn\" (UID: \"a5c9b8dc-3d47-439f-89bf-01f9077f4043\") " pod="openshift-console/downloads-685bfb5f64-bvlqn" Jul 06 23:49:59.727333 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:59.727281 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhrkc\" (UniqueName: \"kubernetes.io/projected/a5c9b8dc-3d47-439f-89bf-01f9077f4043-kube-api-access-zhrkc\") pod \"downloads-685bfb5f64-bvlqn\" (UID: \"a5c9b8dc-3d47-439f-89bf-01f9077f4043\") " pod="openshift-console/downloads-685bfb5f64-bvlqn" Jul 06 23:49:59.866087 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:49:59.865992 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-bvlqn" Jul 06 23:50:00.751729 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:00.751674 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-v978g" Jul 06 23:50:02.742156 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:02.742118 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/450852fa-55ab-4354-8781-e67dd3330970-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-5m4nb\" (UID: \"450852fa-55ab-4354-8781-e67dd3330970\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5m4nb" Jul 06 23:50:02.744917 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:02.744877 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/450852fa-55ab-4354-8781-e67dd3330970-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-5m4nb\" (UID: \"450852fa-55ab-4354-8781-e67dd3330970\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5m4nb" Jul 06 23:50:02.992449 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:02.992393 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xk7b2\"" Jul 06 23:50:02.999532 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:02.999511 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5m4nb" Jul 06 23:50:03.092573 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:03.092412 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-bvlqn"] Jul 06 23:50:03.095320 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:50:03.095247 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5c9b8dc_3d47_439f_89bf_01f9077f4043.slice/crio-fa09871cacd33727b6e2872b35956d6338c14812e666c70ff9548cb1c3f19c30 WatchSource:0}: Error finding container fa09871cacd33727b6e2872b35956d6338c14812e666c70ff9548cb1c3f19c30: Status 404 returned error can't find the container with id fa09871cacd33727b6e2872b35956d6338c14812e666c70ff9548cb1c3f19c30 Jul 06 23:50:03.152243 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:03.152212 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5m4nb"] Jul 06 23:50:03.154961 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:50:03.154927 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod450852fa_55ab_4354_8781_e67dd3330970.slice/crio-58f01518389f552ab5fa98d2906ea341ecd0ae2dfb8eca64fa27986fa0ffd694 WatchSource:0}: Error finding container 58f01518389f552ab5fa98d2906ea341ecd0ae2dfb8eca64fa27986fa0ffd694: Status 404 returned error can't find the container with id 58f01518389f552ab5fa98d2906ea341ecd0ae2dfb8eca64fa27986fa0ffd694 Jul 06 23:50:03.795727 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:03.795691 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5m4nb" event={"ID":"450852fa-55ab-4354-8781-e67dd3330970","Type":"ContainerStarted","Data":"58f01518389f552ab5fa98d2906ea341ecd0ae2dfb8eca64fa27986fa0ffd694"} Jul 06 23:50:03.796957 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:03.796928 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-bvlqn" event={"ID":"a5c9b8dc-3d47-439f-89bf-01f9077f4043","Type":"ContainerStarted","Data":"fa09871cacd33727b6e2872b35956d6338c14812e666c70ff9548cb1c3f19c30"} Jul 06 23:50:03.798632 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:03.798565 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffcc8559-9vmvx" event={"ID":"19494d5f-77e6-449a-b651-d104648d5dd9","Type":"ContainerStarted","Data":"47c1889f95f3a8a91ab8c0551dd731afe4e6188cf226f31695979ab9fb4ff6b7"} Jul 06 23:50:03.798812 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:03.798781 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffcc8559-9vmvx" Jul 06 23:50:03.800427 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:03.800397 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffcc8559-9vmvx" Jul 06 23:50:03.801154 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:03.801130 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rt6cm" event={"ID":"44cf08e2-0939-4411-8f46-7407b0e66e33","Type":"ContainerStarted","Data":"be9fb293082ea7397d0389bbe912cd8d4394195b78a2751f7b00588cf5472262"} Jul 06 23:50:03.818486 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:03.818432 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-85ffcc8559-9vmvx" podStartSLOduration=2.146814304 podStartE2EDuration="6.818417152s" totalImagesPullingTime="4.671602848s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:03.81750777 +0000 UTC m=+86.027237303" watchObservedRunningTime="2026-07-06 23:50:03.818417152 +0000 UTC m=+86.028146675" Jul 06 23:50:03.842943 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:03.842881 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rt6cm" podStartSLOduration=18.361994539 podStartE2EDuration="26.842857708s" totalImagesPullingTime="8.480863169s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:49:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:03.841428217 +0000 UTC m=+86.051157762" watchObservedRunningTime="2026-07-06 23:50:03.842857708 +0000 UTC m=+86.052587231" Jul 06 23:50:05.335089 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:05.335008 2553 patch_prober.go:28] interesting pod/image-registry-7c586d98b6-vr5mc container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 06 23:50:05.335089 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:05.335068 2553 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" podUID="1bc2dc5e-4eab-40a3-9720-ff2ae2746cad" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 23:50:05.544617 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:05.544582 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-pgkjd"] Jul 06 23:50:05.547839 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:05.547822 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-pgkjd" Jul 06 23:50:05.550489 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:05.550465 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 06 23:50:05.550610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:05.550476 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-hgpq8\"" Jul 06 23:50:05.556376 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:05.556352 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-pgkjd"] Jul 06 23:50:05.669333 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:05.669275 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/22edb4ea-f244-4192-af32-adbea1984765-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-pgkjd\" (UID: \"22edb4ea-f244-4192-af32-adbea1984765\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-pgkjd" Jul 06 23:50:05.769886 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:05.769847 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/22edb4ea-f244-4192-af32-adbea1984765-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-pgkjd\" (UID: \"22edb4ea-f244-4192-af32-adbea1984765\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-pgkjd" Jul 06 23:50:05.772665 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:05.772639 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/22edb4ea-f244-4192-af32-adbea1984765-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-pgkjd\" (UID: \"22edb4ea-f244-4192-af32-adbea1984765\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-pgkjd" Jul 06 23:50:05.810227 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:05.810177 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5m4nb" event={"ID":"450852fa-55ab-4354-8781-e67dd3330970","Type":"ContainerStarted","Data":"37e0c0b1b00672478316fc91f555e65400a18c2efa1dd815049f8b74696a0839"} Jul 06 23:50:05.827060 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:05.827006 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5m4nb" podStartSLOduration=33.974808763 podStartE2EDuration="35.826987793s" totalImagesPullingTime="1.85217903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:05.825421872 +0000 UTC m=+88.035151413" watchObservedRunningTime="2026-07-06 23:50:05.826987793 +0000 UTC m=+88.036717313" Jul 06 23:50:05.857190 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:05.857059 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-pgkjd" Jul 06 23:50:06.003444 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:06.003409 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-pgkjd"] Jul 06 23:50:06.007227 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:50:06.007193 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22edb4ea_f244_4192_af32_adbea1984765.slice/crio-7d9577b3bf12ad0bf41352de143f241d8ed26ffeb337320346bdb518a09cc922 WatchSource:0}: Error finding container 7d9577b3bf12ad0bf41352de143f241d8ed26ffeb337320346bdb518a09cc922: Status 404 returned error can't find the container with id 7d9577b3bf12ad0bf41352de143f241d8ed26ffeb337320346bdb518a09cc922 Jul 06 23:50:06.732882 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:06.732851 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:50:06.816937 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:06.816860 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-pgkjd" event={"ID":"22edb4ea-f244-4192-af32-adbea1984765","Type":"ContainerStarted","Data":"7d9577b3bf12ad0bf41352de143f241d8ed26ffeb337320346bdb518a09cc922"} Jul 06 23:50:07.822948 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:07.822907 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-pgkjd" event={"ID":"22edb4ea-f244-4192-af32-adbea1984765","Type":"ContainerStarted","Data":"98a611382439f1ffb0ed88faf9c05253d9a0c34c2157fdf48ad873549fdf87dd"} Jul 06 23:50:07.823437 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:07.823135 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-pgkjd" Jul 06 23:50:07.829134 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:07.829101 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-pgkjd" Jul 06 23:50:07.839087 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:07.839029 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-pgkjd" podStartSLOduration=1.654699474 podStartE2EDuration="2.83900613s" totalImagesPullingTime="1.184306656s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:50:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:07.838254253 +0000 UTC m=+90.047983795" watchObservedRunningTime="2026-07-06 23:50:07.83900613 +0000 UTC m=+90.048735655" Jul 06 23:50:07.867282 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:07.867247 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bc4c4fdbb-9gqpg"] Jul 06 23:50:07.870636 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:07.870606 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bc4c4fdbb-9gqpg" Jul 06 23:50:07.873846 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:07.873656 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 23:50:07.873846 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:07.873720 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 23:50:07.873846 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:07.873739 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 23:50:07.873846 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:07.873762 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 23:50:07.874478 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:07.874134 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 23:50:07.874792 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:07.874765 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5bz9s\"" Jul 06 23:50:07.881988 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:07.881957 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bc4c4fdbb-9gqpg"] Jul 06 23:50:07.990223 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:07.990188 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqqrt\" (UniqueName: \"kubernetes.io/projected/9a7e3b36-ef45-4488-bf49-e1dec85e7174-kube-api-access-hqqrt\") pod \"console-bc4c4fdbb-9gqpg\" (UID: \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\") " pod="openshift-console/console-bc4c4fdbb-9gqpg" Jul 06 23:50:07.990429 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:07.990259 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a7e3b36-ef45-4488-bf49-e1dec85e7174-console-serving-cert\") pod \"console-bc4c4fdbb-9gqpg\" (UID: \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\") " pod="openshift-console/console-bc4c4fdbb-9gqpg" Jul 06 23:50:07.990429 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:07.990399 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a7e3b36-ef45-4488-bf49-e1dec85e7174-oauth-serving-cert\") pod \"console-bc4c4fdbb-9gqpg\" (UID: \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\") " pod="openshift-console/console-bc4c4fdbb-9gqpg" Jul 06 23:50:07.990556 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:07.990431 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a7e3b36-ef45-4488-bf49-e1dec85e7174-console-oauth-config\") pod \"console-bc4c4fdbb-9gqpg\" (UID: \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\") " pod="openshift-console/console-bc4c4fdbb-9gqpg" Jul 06 23:50:07.990556 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:07.990462 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a7e3b36-ef45-4488-bf49-e1dec85e7174-service-ca\") pod \"console-bc4c4fdbb-9gqpg\" (UID: \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\") " pod="openshift-console/console-bc4c4fdbb-9gqpg" Jul 06 23:50:07.990556 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:07.990502 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a7e3b36-ef45-4488-bf49-e1dec85e7174-console-config\") pod \"console-bc4c4fdbb-9gqpg\" (UID: \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\") " pod="openshift-console/console-bc4c4fdbb-9gqpg" Jul 06 23:50:08.091855 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.091768 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a7e3b36-ef45-4488-bf49-e1dec85e7174-console-config\") pod \"console-bc4c4fdbb-9gqpg\" (UID: \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\") " pod="openshift-console/console-bc4c4fdbb-9gqpg" Jul 06 23:50:08.091855 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.091820 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqqrt\" (UniqueName: \"kubernetes.io/projected/9a7e3b36-ef45-4488-bf49-e1dec85e7174-kube-api-access-hqqrt\") pod \"console-bc4c4fdbb-9gqpg\" (UID: \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\") " pod="openshift-console/console-bc4c4fdbb-9gqpg" Jul 06 23:50:08.092116 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.091879 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a7e3b36-ef45-4488-bf49-e1dec85e7174-console-serving-cert\") pod \"console-bc4c4fdbb-9gqpg\" (UID: \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\") " pod="openshift-console/console-bc4c4fdbb-9gqpg" Jul 06 23:50:08.092116 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.091924 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a7e3b36-ef45-4488-bf49-e1dec85e7174-oauth-serving-cert\") pod \"console-bc4c4fdbb-9gqpg\" (UID: \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\") " pod="openshift-console/console-bc4c4fdbb-9gqpg" Jul 06 23:50:08.092116 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.091961 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a7e3b36-ef45-4488-bf49-e1dec85e7174-console-oauth-config\") pod \"console-bc4c4fdbb-9gqpg\" (UID: \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\") " pod="openshift-console/console-bc4c4fdbb-9gqpg" Jul 06 23:50:08.092262 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.092248 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a7e3b36-ef45-4488-bf49-e1dec85e7174-service-ca\") pod \"console-bc4c4fdbb-9gqpg\" (UID: \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\") " pod="openshift-console/console-bc4c4fdbb-9gqpg" Jul 06 23:50:08.092893 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.092858 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a7e3b36-ef45-4488-bf49-e1dec85e7174-console-config\") pod \"console-bc4c4fdbb-9gqpg\" (UID: \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\") " pod="openshift-console/console-bc4c4fdbb-9gqpg" Jul 06 23:50:08.092893 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.092870 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a7e3b36-ef45-4488-bf49-e1dec85e7174-service-ca\") pod \"console-bc4c4fdbb-9gqpg\" (UID: \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\") " pod="openshift-console/console-bc4c4fdbb-9gqpg" Jul 06 23:50:08.093041 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.092863 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a7e3b36-ef45-4488-bf49-e1dec85e7174-oauth-serving-cert\") pod \"console-bc4c4fdbb-9gqpg\" (UID: \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\") " pod="openshift-console/console-bc4c4fdbb-9gqpg" Jul 06 23:50:08.094857 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.094833 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a7e3b36-ef45-4488-bf49-e1dec85e7174-console-serving-cert\") pod \"console-bc4c4fdbb-9gqpg\" (UID: \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\") " pod="openshift-console/console-bc4c4fdbb-9gqpg" Jul 06 23:50:08.094978 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.094919 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a7e3b36-ef45-4488-bf49-e1dec85e7174-console-oauth-config\") pod \"console-bc4c4fdbb-9gqpg\" (UID: \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\") " pod="openshift-console/console-bc4c4fdbb-9gqpg" Jul 06 23:50:08.102163 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.102109 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqqrt\" (UniqueName: \"kubernetes.io/projected/9a7e3b36-ef45-4488-bf49-e1dec85e7174-kube-api-access-hqqrt\") pod \"console-bc4c4fdbb-9gqpg\" (UID: \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\") " pod="openshift-console/console-bc4c4fdbb-9gqpg" Jul 06 23:50:08.182172 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.182131 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bc4c4fdbb-9gqpg" Jul 06 23:50:08.349531 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.349426 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bc4c4fdbb-9gqpg"] Jul 06 23:50:08.352758 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:50:08.352720 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a7e3b36_ef45_4488_bf49_e1dec85e7174.slice/crio-86a10f4e5b976e4113475dc7813fc6fcc730c6a3ebb34eb2d808a3fe5d64e416 WatchSource:0}: Error finding container 86a10f4e5b976e4113475dc7813fc6fcc730c6a3ebb34eb2d808a3fe5d64e416: Status 404 returned error can't find the container with id 86a10f4e5b976e4113475dc7813fc6fcc730c6a3ebb34eb2d808a3fe5d64e416 Jul 06 23:50:08.601245 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.601165 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-xswqw"] Jul 06 23:50:08.605626 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.605604 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-xswqw" Jul 06 23:50:08.608696 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.608669 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 06 23:50:08.608826 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.608669 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 06 23:50:08.609881 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.609859 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 23:50:08.609978 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.609926 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-b4z42\"" Jul 06 23:50:08.619448 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.619252 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-xswqw"] Jul 06 23:50:08.697568 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.697519 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45b28db1-4de7-47eb-83b3-5f728373814f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-xswqw\" (UID: \"45b28db1-4de7-47eb-83b3-5f728373814f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-xswqw" Jul 06 23:50:08.697760 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.697581 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45b28db1-4de7-47eb-83b3-5f728373814f-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-xswqw\" (UID: \"45b28db1-4de7-47eb-83b3-5f728373814f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-xswqw" Jul 06 23:50:08.697760 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.697608 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b56zd\" (UniqueName: \"kubernetes.io/projected/45b28db1-4de7-47eb-83b3-5f728373814f-kube-api-access-b56zd\") pod \"prometheus-operator-567f7b8b7c-xswqw\" (UID: \"45b28db1-4de7-47eb-83b3-5f728373814f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-xswqw" Jul 06 23:50:08.697760 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.697633 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/45b28db1-4de7-47eb-83b3-5f728373814f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-xswqw\" (UID: \"45b28db1-4de7-47eb-83b3-5f728373814f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-xswqw" Jul 06 23:50:08.798196 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.798160 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45b28db1-4de7-47eb-83b3-5f728373814f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-xswqw\" (UID: \"45b28db1-4de7-47eb-83b3-5f728373814f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-xswqw" Jul 06 23:50:08.798401 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.798217 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45b28db1-4de7-47eb-83b3-5f728373814f-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-xswqw\" (UID: \"45b28db1-4de7-47eb-83b3-5f728373814f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-xswqw" Jul 06 23:50:08.798401 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.798261 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b56zd\" (UniqueName: \"kubernetes.io/projected/45b28db1-4de7-47eb-83b3-5f728373814f-kube-api-access-b56zd\") pod \"prometheus-operator-567f7b8b7c-xswqw\" (UID: \"45b28db1-4de7-47eb-83b3-5f728373814f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-xswqw" Jul 06 23:50:08.798401 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.798285 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/45b28db1-4de7-47eb-83b3-5f728373814f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-xswqw\" (UID: \"45b28db1-4de7-47eb-83b3-5f728373814f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-xswqw" Jul 06 23:50:08.798565 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:50:08.798506 2553 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 06 23:50:08.798615 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:50:08.798575 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45b28db1-4de7-47eb-83b3-5f728373814f-prometheus-operator-tls podName:45b28db1-4de7-47eb-83b3-5f728373814f nodeName:}" failed. No retries permitted until 2026-07-06 23:50:09.298554572 +0000 UTC m=+91.508284074 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/45b28db1-4de7-47eb-83b3-5f728373814f-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-xswqw" (UID: "45b28db1-4de7-47eb-83b3-5f728373814f") : secret "prometheus-operator-tls" not found Jul 06 23:50:08.799260 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.799233 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45b28db1-4de7-47eb-83b3-5f728373814f-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-xswqw\" (UID: \"45b28db1-4de7-47eb-83b3-5f728373814f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-xswqw" Jul 06 23:50:08.801306 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.801265 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45b28db1-4de7-47eb-83b3-5f728373814f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-xswqw\" (UID: \"45b28db1-4de7-47eb-83b3-5f728373814f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-xswqw" Jul 06 23:50:08.810891 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.810857 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b56zd\" (UniqueName: \"kubernetes.io/projected/45b28db1-4de7-47eb-83b3-5f728373814f-kube-api-access-b56zd\") pod \"prometheus-operator-567f7b8b7c-xswqw\" (UID: \"45b28db1-4de7-47eb-83b3-5f728373814f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-xswqw" Jul 06 23:50:08.829165 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:08.829127 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bc4c4fdbb-9gqpg" event={"ID":"9a7e3b36-ef45-4488-bf49-e1dec85e7174","Type":"ContainerStarted","Data":"86a10f4e5b976e4113475dc7813fc6fcc730c6a3ebb34eb2d808a3fe5d64e416"} Jul 06 23:50:09.301847 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:09.301725 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/45b28db1-4de7-47eb-83b3-5f728373814f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-xswqw\" (UID: \"45b28db1-4de7-47eb-83b3-5f728373814f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-xswqw" Jul 06 23:50:09.304795 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:09.304729 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/45b28db1-4de7-47eb-83b3-5f728373814f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-xswqw\" (UID: \"45b28db1-4de7-47eb-83b3-5f728373814f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-xswqw" Jul 06 23:50:09.522806 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:09.522769 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-xswqw" Jul 06 23:50:09.681166 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:09.681126 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-xswqw"] Jul 06 23:50:09.686808 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:50:09.686776 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45b28db1_4de7_47eb_83b3_5f728373814f.slice/crio-094b14f1b00af450317995420768fa01a46089d1fa2696329a2c9c06901d9d4d WatchSource:0}: Error finding container 094b14f1b00af450317995420768fa01a46089d1fa2696329a2c9c06901d9d4d: Status 404 returned error can't find the container with id 094b14f1b00af450317995420768fa01a46089d1fa2696329a2c9c06901d9d4d Jul 06 23:50:09.835142 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:09.835044 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-xswqw" event={"ID":"45b28db1-4de7-47eb-83b3-5f728373814f","Type":"ContainerStarted","Data":"094b14f1b00af450317995420768fa01a46089d1fa2696329a2c9c06901d9d4d"} Jul 06 23:50:12.853379 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:12.853333 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-xswqw" event={"ID":"45b28db1-4de7-47eb-83b3-5f728373814f","Type":"ContainerStarted","Data":"8819cb7ce547c19632c849831d9cd2ec509a1b7d3533a2627ab11048780b51ee"} Jul 06 23:50:12.853849 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:12.853407 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-xswqw" event={"ID":"45b28db1-4de7-47eb-83b3-5f728373814f","Type":"ContainerStarted","Data":"d126a6ddcd17581ffb762238ad39c94a2d5c77f82e9313666044e57d067a2aaa"} Jul 06 23:50:12.855408 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:12.855380 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bc4c4fdbb-9gqpg" event={"ID":"9a7e3b36-ef45-4488-bf49-e1dec85e7174","Type":"ContainerStarted","Data":"311fe0546b9d957c52573d6c1674334162fb87892ec335f45245f962602f25a5"} Jul 06 23:50:12.872893 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:12.872831 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-xswqw" podStartSLOduration=2.792102873 podStartE2EDuration="4.872815332s" totalImagesPullingTime="2.080712459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:50:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:12.871223159 +0000 UTC m=+95.080952700" watchObservedRunningTime="2026-07-06 23:50:12.872815332 +0000 UTC m=+95.082544852" Jul 06 23:50:12.891190 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:12.891130 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bc4c4fdbb-9gqpg" podStartSLOduration=2.476263967 podStartE2EDuration="5.891110092s" totalImagesPullingTime="3.414846125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:50:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:12.88984978 +0000 UTC m=+95.099579302" watchObservedRunningTime="2026-07-06 23:50:12.891110092 +0000 UTC m=+95.100839682" Jul 06 23:50:15.005422 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.005381 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kxl2w"] Jul 06 23:50:15.012183 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.012152 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kxl2w" Jul 06 23:50:15.015037 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.015004 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-s94tz\"" Jul 06 23:50:15.016352 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.016273 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 06 23:50:15.016352 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.016287 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 06 23:50:15.016637 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.016288 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 06 23:50:15.018455 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.018381 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xg2fq"] Jul 06 23:50:15.022674 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.022586 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kxl2w"] Jul 06 23:50:15.022781 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.022658 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:15.025441 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.025412 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 23:50:15.025595 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.025570 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 23:50:15.025657 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.025647 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hhxhk\"" Jul 06 23:50:15.025809 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.025782 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 23:50:15.153795 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.153749 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc4f8\" (UniqueName: \"kubernetes.io/projected/376d341d-f1b9-4d45-90e8-4e77fe4f9837-kube-api-access-wc4f8\") pod \"node-exporter-xg2fq\" (UID: \"376d341d-f1b9-4d45-90e8-4e77fe4f9837\") " pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:15.154040 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.153839 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z2bw\" (UniqueName: \"kubernetes.io/projected/4b569175-860d-44bc-841d-a268294a112d-kube-api-access-4z2bw\") pod \"kube-state-metrics-7c9dbb8b7d-kxl2w\" (UID: \"4b569175-860d-44bc-841d-a268294a112d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kxl2w" Jul 06 23:50:15.154040 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.153917 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/376d341d-f1b9-4d45-90e8-4e77fe4f9837-node-exporter-textfile\") pod \"node-exporter-xg2fq\" (UID: \"376d341d-f1b9-4d45-90e8-4e77fe4f9837\") " pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:15.154040 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.153955 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/376d341d-f1b9-4d45-90e8-4e77fe4f9837-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xg2fq\" (UID: \"376d341d-f1b9-4d45-90e8-4e77fe4f9837\") " pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:15.154040 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.153996 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4b569175-860d-44bc-841d-a268294a112d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-kxl2w\" (UID: \"4b569175-860d-44bc-841d-a268294a112d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kxl2w" Jul 06 23:50:15.154245 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.154055 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/376d341d-f1b9-4d45-90e8-4e77fe4f9837-metrics-client-ca\") pod \"node-exporter-xg2fq\" (UID: \"376d341d-f1b9-4d45-90e8-4e77fe4f9837\") " pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:15.154245 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.154112 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4b569175-860d-44bc-841d-a268294a112d-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-kxl2w\" (UID: \"4b569175-860d-44bc-841d-a268294a112d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kxl2w" Jul 06 23:50:15.154245 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.154143 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b569175-860d-44bc-841d-a268294a112d-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-kxl2w\" (UID: \"4b569175-860d-44bc-841d-a268294a112d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kxl2w" Jul 06 23:50:15.154245 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.154181 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b569175-860d-44bc-841d-a268294a112d-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-kxl2w\" (UID: \"4b569175-860d-44bc-841d-a268294a112d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kxl2w" Jul 06 23:50:15.154245 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.154209 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/376d341d-f1b9-4d45-90e8-4e77fe4f9837-node-exporter-wtmp\") pod \"node-exporter-xg2fq\" (UID: \"376d341d-f1b9-4d45-90e8-4e77fe4f9837\") " pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:15.154245 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.154239 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/376d341d-f1b9-4d45-90e8-4e77fe4f9837-root\") pod \"node-exporter-xg2fq\" (UID: \"376d341d-f1b9-4d45-90e8-4e77fe4f9837\") " pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:15.154545 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.154340 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/376d341d-f1b9-4d45-90e8-4e77fe4f9837-sys\") pod \"node-exporter-xg2fq\" (UID: \"376d341d-f1b9-4d45-90e8-4e77fe4f9837\") " pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:15.154545 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.154389 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4b569175-860d-44bc-841d-a268294a112d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-kxl2w\" (UID: \"4b569175-860d-44bc-841d-a268294a112d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kxl2w" Jul 06 23:50:15.154545 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.154421 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/376d341d-f1b9-4d45-90e8-4e77fe4f9837-node-exporter-tls\") pod \"node-exporter-xg2fq\" (UID: \"376d341d-f1b9-4d45-90e8-4e77fe4f9837\") " pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:15.154545 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.154450 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/376d341d-f1b9-4d45-90e8-4e77fe4f9837-node-exporter-accelerators-collector-config\") pod \"node-exporter-xg2fq\" (UID: \"376d341d-f1b9-4d45-90e8-4e77fe4f9837\") " pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:15.255553 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.255457 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/376d341d-f1b9-4d45-90e8-4e77fe4f9837-sys\") pod \"node-exporter-xg2fq\" (UID: \"376d341d-f1b9-4d45-90e8-4e77fe4f9837\") " pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:15.255553 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.255508 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4b569175-860d-44bc-841d-a268294a112d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-kxl2w\" (UID: \"4b569175-860d-44bc-841d-a268294a112d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kxl2w" Jul 06 23:50:15.255553 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.255544 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/376d341d-f1b9-4d45-90e8-4e77fe4f9837-node-exporter-tls\") pod \"node-exporter-xg2fq\" (UID: \"376d341d-f1b9-4d45-90e8-4e77fe4f9837\") " pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:15.255852 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.255574 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/376d341d-f1b9-4d45-90e8-4e77fe4f9837-node-exporter-accelerators-collector-config\") pod \"node-exporter-xg2fq\" (UID: \"376d341d-f1b9-4d45-90e8-4e77fe4f9837\") " pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:15.255852 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.255595 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/376d341d-f1b9-4d45-90e8-4e77fe4f9837-sys\") pod \"node-exporter-xg2fq\" (UID: \"376d341d-f1b9-4d45-90e8-4e77fe4f9837\") " pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:15.255852 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.255622 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc4f8\" (UniqueName: \"kubernetes.io/projected/376d341d-f1b9-4d45-90e8-4e77fe4f9837-kube-api-access-wc4f8\") pod \"node-exporter-xg2fq\" (UID: \"376d341d-f1b9-4d45-90e8-4e77fe4f9837\") " pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:15.255852 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.255690 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z2bw\" (UniqueName: \"kubernetes.io/projected/4b569175-860d-44bc-841d-a268294a112d-kube-api-access-4z2bw\") pod \"kube-state-metrics-7c9dbb8b7d-kxl2w\" (UID: \"4b569175-860d-44bc-841d-a268294a112d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kxl2w" Jul 06 23:50:15.255852 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.255713 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/376d341d-f1b9-4d45-90e8-4e77fe4f9837-node-exporter-textfile\") pod \"node-exporter-xg2fq\" (UID: \"376d341d-f1b9-4d45-90e8-4e77fe4f9837\") " pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:15.255852 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.255729 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/376d341d-f1b9-4d45-90e8-4e77fe4f9837-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xg2fq\" (UID: \"376d341d-f1b9-4d45-90e8-4e77fe4f9837\") " pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:15.255852 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:50:15.255738 2553 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 06 23:50:15.255852 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.255764 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4b569175-860d-44bc-841d-a268294a112d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-kxl2w\" (UID: \"4b569175-860d-44bc-841d-a268294a112d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kxl2w" Jul 06 23:50:15.255852 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:50:15.255804 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/376d341d-f1b9-4d45-90e8-4e77fe4f9837-node-exporter-tls podName:376d341d-f1b9-4d45-90e8-4e77fe4f9837 nodeName:}" failed. No retries permitted until 2026-07-06 23:50:15.755783621 +0000 UTC m=+97.965513131 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/376d341d-f1b9-4d45-90e8-4e77fe4f9837-node-exporter-tls") pod "node-exporter-xg2fq" (UID: "376d341d-f1b9-4d45-90e8-4e77fe4f9837") : secret "node-exporter-tls" not found Jul 06 23:50:15.255852 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.255838 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/376d341d-f1b9-4d45-90e8-4e77fe4f9837-metrics-client-ca\") pod \"node-exporter-xg2fq\" (UID: \"376d341d-f1b9-4d45-90e8-4e77fe4f9837\") " pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:15.256373 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.255874 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4b569175-860d-44bc-841d-a268294a112d-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-kxl2w\" (UID: \"4b569175-860d-44bc-841d-a268294a112d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kxl2w" Jul 06 23:50:15.256373 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.255898 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b569175-860d-44bc-841d-a268294a112d-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-kxl2w\" (UID: \"4b569175-860d-44bc-841d-a268294a112d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kxl2w" Jul 06 23:50:15.256373 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.256250 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4b569175-860d-44bc-841d-a268294a112d-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-kxl2w\" (UID: \"4b569175-860d-44bc-841d-a268294a112d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kxl2w" Jul 06 23:50:15.256373 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.256262 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/376d341d-f1b9-4d45-90e8-4e77fe4f9837-node-exporter-textfile\") pod \"node-exporter-xg2fq\" (UID: \"376d341d-f1b9-4d45-90e8-4e77fe4f9837\") " pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:15.256373 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.256331 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/376d341d-f1b9-4d45-90e8-4e77fe4f9837-node-exporter-accelerators-collector-config\") pod \"node-exporter-xg2fq\" (UID: \"376d341d-f1b9-4d45-90e8-4e77fe4f9837\") " pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:15.256614 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.256378 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b569175-860d-44bc-841d-a268294a112d-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-kxl2w\" (UID: \"4b569175-860d-44bc-841d-a268294a112d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kxl2w" Jul 06 23:50:15.256614 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.256406 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/376d341d-f1b9-4d45-90e8-4e77fe4f9837-node-exporter-wtmp\") pod \"node-exporter-xg2fq\" (UID: \"376d341d-f1b9-4d45-90e8-4e77fe4f9837\") " pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:15.256614 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.256436 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/376d341d-f1b9-4d45-90e8-4e77fe4f9837-root\") pod \"node-exporter-xg2fq\" (UID: \"376d341d-f1b9-4d45-90e8-4e77fe4f9837\") " pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:15.256614 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.256452 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/376d341d-f1b9-4d45-90e8-4e77fe4f9837-metrics-client-ca\") pod \"node-exporter-xg2fq\" (UID: \"376d341d-f1b9-4d45-90e8-4e77fe4f9837\") " pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:15.256614 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.256516 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/376d341d-f1b9-4d45-90e8-4e77fe4f9837-root\") pod \"node-exporter-xg2fq\" (UID: \"376d341d-f1b9-4d45-90e8-4e77fe4f9837\") " pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:15.256614 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.256534 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4b569175-860d-44bc-841d-a268294a112d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-kxl2w\" (UID: \"4b569175-860d-44bc-841d-a268294a112d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kxl2w" Jul 06 23:50:15.256614 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.256601 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/376d341d-f1b9-4d45-90e8-4e77fe4f9837-node-exporter-wtmp\") pod \"node-exporter-xg2fq\" (UID: \"376d341d-f1b9-4d45-90e8-4e77fe4f9837\") " pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:15.257028 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.257005 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b569175-860d-44bc-841d-a268294a112d-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-kxl2w\" (UID: \"4b569175-860d-44bc-841d-a268294a112d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kxl2w" Jul 06 23:50:15.258951 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.258903 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4b569175-860d-44bc-841d-a268294a112d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-kxl2w\" (UID: \"4b569175-860d-44bc-841d-a268294a112d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kxl2w" Jul 06 23:50:15.259104 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.259078 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/376d341d-f1b9-4d45-90e8-4e77fe4f9837-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xg2fq\" (UID: \"376d341d-f1b9-4d45-90e8-4e77fe4f9837\") " pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:15.259184 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.259096 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b569175-860d-44bc-841d-a268294a112d-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-kxl2w\" (UID: \"4b569175-860d-44bc-841d-a268294a112d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kxl2w" Jul 06 23:50:15.265182 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.265154 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc4f8\" (UniqueName: \"kubernetes.io/projected/376d341d-f1b9-4d45-90e8-4e77fe4f9837-kube-api-access-wc4f8\") pod \"node-exporter-xg2fq\" (UID: \"376d341d-f1b9-4d45-90e8-4e77fe4f9837\") " pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:15.265385 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.265363 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z2bw\" (UniqueName: \"kubernetes.io/projected/4b569175-860d-44bc-841d-a268294a112d-kube-api-access-4z2bw\") pod \"kube-state-metrics-7c9dbb8b7d-kxl2w\" (UID: \"4b569175-860d-44bc-841d-a268294a112d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kxl2w" Jul 06 23:50:15.324547 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.324512 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kxl2w" Jul 06 23:50:15.762697 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.762591 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/376d341d-f1b9-4d45-90e8-4e77fe4f9837-node-exporter-tls\") pod \"node-exporter-xg2fq\" (UID: \"376d341d-f1b9-4d45-90e8-4e77fe4f9837\") " pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:15.765542 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.765414 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/376d341d-f1b9-4d45-90e8-4e77fe4f9837-node-exporter-tls\") pod \"node-exporter-xg2fq\" (UID: \"376d341d-f1b9-4d45-90e8-4e77fe4f9837\") " pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:15.934630 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:15.934594 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xg2fq" Jul 06 23:50:18.183020 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:18.182980 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bc4c4fdbb-9gqpg" Jul 06 23:50:18.183500 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:18.183092 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bc4c4fdbb-9gqpg" Jul 06 23:50:18.189012 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:18.188981 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bc4c4fdbb-9gqpg" Jul 06 23:50:18.884191 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:18.884152 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bc4c4fdbb-9gqpg" Jul 06 23:50:19.038116 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.038080 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7666df4778-bvfdz"] Jul 06 23:50:19.174311 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.174250 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7666df4778-bvfdz"] Jul 06 23:50:19.174493 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.174322 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7666df4778-bvfdz" Jul 06 23:50:19.183628 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.183368 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 23:50:19.193393 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.193275 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0bb271dc-a651-471e-8d03-12423e38f85d-service-ca\") pod \"console-7666df4778-bvfdz\" (UID: \"0bb271dc-a651-471e-8d03-12423e38f85d\") " pod="openshift-console/console-7666df4778-bvfdz" Jul 06 23:50:19.193393 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.193374 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0bb271dc-a651-471e-8d03-12423e38f85d-oauth-serving-cert\") pod \"console-7666df4778-bvfdz\" (UID: \"0bb271dc-a651-471e-8d03-12423e38f85d\") " pod="openshift-console/console-7666df4778-bvfdz" Jul 06 23:50:19.193619 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.193430 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfx5g\" (UniqueName: \"kubernetes.io/projected/0bb271dc-a651-471e-8d03-12423e38f85d-kube-api-access-hfx5g\") pod \"console-7666df4778-bvfdz\" (UID: \"0bb271dc-a651-471e-8d03-12423e38f85d\") " pod="openshift-console/console-7666df4778-bvfdz" Jul 06 23:50:19.193619 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.193478 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0bb271dc-a651-471e-8d03-12423e38f85d-trusted-ca-bundle\") pod \"console-7666df4778-bvfdz\" (UID: \"0bb271dc-a651-471e-8d03-12423e38f85d\") " pod="openshift-console/console-7666df4778-bvfdz" Jul 06 23:50:19.193619 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.193502 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0bb271dc-a651-471e-8d03-12423e38f85d-console-oauth-config\") pod \"console-7666df4778-bvfdz\" (UID: \"0bb271dc-a651-471e-8d03-12423e38f85d\") " pod="openshift-console/console-7666df4778-bvfdz" Jul 06 23:50:19.193619 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.193583 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0bb271dc-a651-471e-8d03-12423e38f85d-console-serving-cert\") pod \"console-7666df4778-bvfdz\" (UID: \"0bb271dc-a651-471e-8d03-12423e38f85d\") " pod="openshift-console/console-7666df4778-bvfdz" Jul 06 23:50:19.193619 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.193610 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0bb271dc-a651-471e-8d03-12423e38f85d-console-config\") pod \"console-7666df4778-bvfdz\" (UID: \"0bb271dc-a651-471e-8d03-12423e38f85d\") " pod="openshift-console/console-7666df4778-bvfdz" Jul 06 23:50:19.273552 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.273516 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-f6cf5fb68-c5vnf"] Jul 06 23:50:19.294281 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.294246 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0bb271dc-a651-471e-8d03-12423e38f85d-console-serving-cert\") pod \"console-7666df4778-bvfdz\" (UID: \"0bb271dc-a651-471e-8d03-12423e38f85d\") " pod="openshift-console/console-7666df4778-bvfdz" Jul 06 23:50:19.294563 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.294344 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0bb271dc-a651-471e-8d03-12423e38f85d-console-config\") pod \"console-7666df4778-bvfdz\" (UID: \"0bb271dc-a651-471e-8d03-12423e38f85d\") " pod="openshift-console/console-7666df4778-bvfdz" Jul 06 23:50:19.294563 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.294398 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0bb271dc-a651-471e-8d03-12423e38f85d-service-ca\") pod \"console-7666df4778-bvfdz\" (UID: \"0bb271dc-a651-471e-8d03-12423e38f85d\") " pod="openshift-console/console-7666df4778-bvfdz" Jul 06 23:50:19.294563 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.294441 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0bb271dc-a651-471e-8d03-12423e38f85d-oauth-serving-cert\") pod \"console-7666df4778-bvfdz\" (UID: \"0bb271dc-a651-471e-8d03-12423e38f85d\") " pod="openshift-console/console-7666df4778-bvfdz" Jul 06 23:50:19.294819 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.294561 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfx5g\" (UniqueName: \"kubernetes.io/projected/0bb271dc-a651-471e-8d03-12423e38f85d-kube-api-access-hfx5g\") pod \"console-7666df4778-bvfdz\" (UID: \"0bb271dc-a651-471e-8d03-12423e38f85d\") " pod="openshift-console/console-7666df4778-bvfdz" Jul 06 23:50:19.294819 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.294611 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0bb271dc-a651-471e-8d03-12423e38f85d-trusted-ca-bundle\") pod \"console-7666df4778-bvfdz\" (UID: \"0bb271dc-a651-471e-8d03-12423e38f85d\") " pod="openshift-console/console-7666df4778-bvfdz" Jul 06 23:50:19.294819 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.294636 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0bb271dc-a651-471e-8d03-12423e38f85d-console-oauth-config\") pod \"console-7666df4778-bvfdz\" (UID: \"0bb271dc-a651-471e-8d03-12423e38f85d\") " pod="openshift-console/console-7666df4778-bvfdz" Jul 06 23:50:19.296005 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.295061 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-f6cf5fb68-c5vnf"] Jul 06 23:50:19.296005 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.295182 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" Jul 06 23:50:19.296005 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.295247 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0bb271dc-a651-471e-8d03-12423e38f85d-service-ca\") pod \"console-7666df4778-bvfdz\" (UID: \"0bb271dc-a651-471e-8d03-12423e38f85d\") " pod="openshift-console/console-7666df4778-bvfdz" Jul 06 23:50:19.296321 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.296022 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0bb271dc-a651-471e-8d03-12423e38f85d-oauth-serving-cert\") pod \"console-7666df4778-bvfdz\" (UID: \"0bb271dc-a651-471e-8d03-12423e38f85d\") " pod="openshift-console/console-7666df4778-bvfdz" Jul 06 23:50:19.296321 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.296192 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0bb271dc-a651-471e-8d03-12423e38f85d-console-config\") pod \"console-7666df4778-bvfdz\" (UID: \"0bb271dc-a651-471e-8d03-12423e38f85d\") " pod="openshift-console/console-7666df4778-bvfdz" Jul 06 23:50:19.297820 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.297797 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0bb271dc-a651-471e-8d03-12423e38f85d-trusted-ca-bundle\") pod \"console-7666df4778-bvfdz\" (UID: \"0bb271dc-a651-471e-8d03-12423e38f85d\") " pod="openshift-console/console-7666df4778-bvfdz" Jul 06 23:50:19.298055 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.297876 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0bb271dc-a651-471e-8d03-12423e38f85d-console-serving-cert\") pod \"console-7666df4778-bvfdz\" (UID: \"0bb271dc-a651-471e-8d03-12423e38f85d\") " pod="openshift-console/console-7666df4778-bvfdz" Jul 06 23:50:19.298198 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.298180 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 06 23:50:19.299348 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.299265 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 06 23:50:19.299348 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.299283 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-bd6sv\"" Jul 06 23:50:19.299507 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.299344 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 06 23:50:19.299635 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.299615 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ejgjhc151ga3h\"" Jul 06 23:50:19.299711 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.299692 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 06 23:50:19.300422 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.300401 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0bb271dc-a651-471e-8d03-12423e38f85d-console-oauth-config\") pod \"console-7666df4778-bvfdz\" (UID: \"0bb271dc-a651-471e-8d03-12423e38f85d\") " pod="openshift-console/console-7666df4778-bvfdz" Jul 06 23:50:19.304464 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.304368 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfx5g\" (UniqueName: \"kubernetes.io/projected/0bb271dc-a651-471e-8d03-12423e38f85d-kube-api-access-hfx5g\") pod \"console-7666df4778-bvfdz\" (UID: \"0bb271dc-a651-471e-8d03-12423e38f85d\") " pod="openshift-console/console-7666df4778-bvfdz" Jul 06 23:50:19.395103 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.395062 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/503b39e8-97e2-4ad8-a4c5-0a716cd2061a-audit-log\") pod \"metrics-server-f6cf5fb68-c5vnf\" (UID: \"503b39e8-97e2-4ad8-a4c5-0a716cd2061a\") " pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" Jul 06 23:50:19.395317 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.395110 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/503b39e8-97e2-4ad8-a4c5-0a716cd2061a-metrics-server-audit-profiles\") pod \"metrics-server-f6cf5fb68-c5vnf\" (UID: \"503b39e8-97e2-4ad8-a4c5-0a716cd2061a\") " pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" Jul 06 23:50:19.395317 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.395197 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxmh8\" (UniqueName: \"kubernetes.io/projected/503b39e8-97e2-4ad8-a4c5-0a716cd2061a-kube-api-access-sxmh8\") pod \"metrics-server-f6cf5fb68-c5vnf\" (UID: \"503b39e8-97e2-4ad8-a4c5-0a716cd2061a\") " pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" Jul 06 23:50:19.395317 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.395238 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/503b39e8-97e2-4ad8-a4c5-0a716cd2061a-secret-metrics-server-client-certs\") pod \"metrics-server-f6cf5fb68-c5vnf\" (UID: \"503b39e8-97e2-4ad8-a4c5-0a716cd2061a\") " pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" Jul 06 23:50:19.395317 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.395269 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503b39e8-97e2-4ad8-a4c5-0a716cd2061a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-f6cf5fb68-c5vnf\" (UID: \"503b39e8-97e2-4ad8-a4c5-0a716cd2061a\") " pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" Jul 06 23:50:19.395513 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.395352 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/503b39e8-97e2-4ad8-a4c5-0a716cd2061a-client-ca-bundle\") pod \"metrics-server-f6cf5fb68-c5vnf\" (UID: \"503b39e8-97e2-4ad8-a4c5-0a716cd2061a\") " pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" Jul 06 23:50:19.395513 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.395478 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/503b39e8-97e2-4ad8-a4c5-0a716cd2061a-secret-metrics-server-tls\") pod \"metrics-server-f6cf5fb68-c5vnf\" (UID: \"503b39e8-97e2-4ad8-a4c5-0a716cd2061a\") " pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" Jul 06 23:50:19.486861 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.486779 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7666df4778-bvfdz" Jul 06 23:50:19.496206 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.496167 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/503b39e8-97e2-4ad8-a4c5-0a716cd2061a-audit-log\") pod \"metrics-server-f6cf5fb68-c5vnf\" (UID: \"503b39e8-97e2-4ad8-a4c5-0a716cd2061a\") " pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" Jul 06 23:50:19.496419 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.496217 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/503b39e8-97e2-4ad8-a4c5-0a716cd2061a-metrics-server-audit-profiles\") pod \"metrics-server-f6cf5fb68-c5vnf\" (UID: \"503b39e8-97e2-4ad8-a4c5-0a716cd2061a\") " pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" Jul 06 23:50:19.496419 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.496254 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxmh8\" (UniqueName: \"kubernetes.io/projected/503b39e8-97e2-4ad8-a4c5-0a716cd2061a-kube-api-access-sxmh8\") pod \"metrics-server-f6cf5fb68-c5vnf\" (UID: \"503b39e8-97e2-4ad8-a4c5-0a716cd2061a\") " pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" Jul 06 23:50:19.496419 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.496289 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/503b39e8-97e2-4ad8-a4c5-0a716cd2061a-secret-metrics-server-client-certs\") pod \"metrics-server-f6cf5fb68-c5vnf\" (UID: \"503b39e8-97e2-4ad8-a4c5-0a716cd2061a\") " pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" Jul 06 23:50:19.496419 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.496337 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503b39e8-97e2-4ad8-a4c5-0a716cd2061a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-f6cf5fb68-c5vnf\" (UID: \"503b39e8-97e2-4ad8-a4c5-0a716cd2061a\") " pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" Jul 06 23:50:19.496419 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.496373 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/503b39e8-97e2-4ad8-a4c5-0a716cd2061a-client-ca-bundle\") pod \"metrics-server-f6cf5fb68-c5vnf\" (UID: \"503b39e8-97e2-4ad8-a4c5-0a716cd2061a\") " pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" Jul 06 23:50:19.496635 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.496458 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/503b39e8-97e2-4ad8-a4c5-0a716cd2061a-secret-metrics-server-tls\") pod \"metrics-server-f6cf5fb68-c5vnf\" (UID: \"503b39e8-97e2-4ad8-a4c5-0a716cd2061a\") " pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" Jul 06 23:50:19.496679 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.496645 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/503b39e8-97e2-4ad8-a4c5-0a716cd2061a-audit-log\") pod \"metrics-server-f6cf5fb68-c5vnf\" (UID: \"503b39e8-97e2-4ad8-a4c5-0a716cd2061a\") " pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" Jul 06 23:50:19.497380 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.497288 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503b39e8-97e2-4ad8-a4c5-0a716cd2061a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-f6cf5fb68-c5vnf\" (UID: \"503b39e8-97e2-4ad8-a4c5-0a716cd2061a\") " pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" Jul 06 23:50:19.497511 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.497466 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/503b39e8-97e2-4ad8-a4c5-0a716cd2061a-metrics-server-audit-profiles\") pod \"metrics-server-f6cf5fb68-c5vnf\" (UID: \"503b39e8-97e2-4ad8-a4c5-0a716cd2061a\") " pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" Jul 06 23:50:19.499395 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.499359 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/503b39e8-97e2-4ad8-a4c5-0a716cd2061a-secret-metrics-server-tls\") pod \"metrics-server-f6cf5fb68-c5vnf\" (UID: \"503b39e8-97e2-4ad8-a4c5-0a716cd2061a\") " pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" Jul 06 23:50:19.500342 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.500191 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/503b39e8-97e2-4ad8-a4c5-0a716cd2061a-client-ca-bundle\") pod \"metrics-server-f6cf5fb68-c5vnf\" (UID: \"503b39e8-97e2-4ad8-a4c5-0a716cd2061a\") " pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" Jul 06 23:50:19.500342 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.500215 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/503b39e8-97e2-4ad8-a4c5-0a716cd2061a-secret-metrics-server-client-certs\") pod \"metrics-server-f6cf5fb68-c5vnf\" (UID: \"503b39e8-97e2-4ad8-a4c5-0a716cd2061a\") " pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" Jul 06 23:50:19.504879 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.504747 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxmh8\" (UniqueName: \"kubernetes.io/projected/503b39e8-97e2-4ad8-a4c5-0a716cd2061a-kube-api-access-sxmh8\") pod \"metrics-server-f6cf5fb68-c5vnf\" (UID: \"503b39e8-97e2-4ad8-a4c5-0a716cd2061a\") " pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" Jul 06 23:50:19.624103 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.624057 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" Jul 06 23:50:19.727835 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.726805 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-rtj7l"] Jul 06 23:50:19.749048 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.748960 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-rtj7l"] Jul 06 23:50:19.749223 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.749093 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-rtj7l" Jul 06 23:50:19.751778 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.751753 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 06 23:50:19.751937 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.751800 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-f2vsq\"" Jul 06 23:50:19.799983 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.799942 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvsd2\" (UniqueName: \"kubernetes.io/projected/229ece0b-6bfb-4628-bb44-91087ba93096-kube-api-access-nvsd2\") pod \"monitoring-plugin-d5dcff7cf-rtj7l\" (UID: \"229ece0b-6bfb-4628-bb44-91087ba93096\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-rtj7l" Jul 06 23:50:19.800199 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.800024 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/229ece0b-6bfb-4628-bb44-91087ba93096-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-rtj7l\" (UID: \"229ece0b-6bfb-4628-bb44-91087ba93096\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-rtj7l" Jul 06 23:50:19.901319 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.901271 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/229ece0b-6bfb-4628-bb44-91087ba93096-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-rtj7l\" (UID: \"229ece0b-6bfb-4628-bb44-91087ba93096\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-rtj7l" Jul 06 23:50:19.901877 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.901529 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvsd2\" (UniqueName: \"kubernetes.io/projected/229ece0b-6bfb-4628-bb44-91087ba93096-kube-api-access-nvsd2\") pod \"monitoring-plugin-d5dcff7cf-rtj7l\" (UID: \"229ece0b-6bfb-4628-bb44-91087ba93096\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-rtj7l" Jul 06 23:50:19.903916 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.903884 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/229ece0b-6bfb-4628-bb44-91087ba93096-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-rtj7l\" (UID: \"229ece0b-6bfb-4628-bb44-91087ba93096\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-rtj7l" Jul 06 23:50:19.910961 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:19.910920 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvsd2\" (UniqueName: \"kubernetes.io/projected/229ece0b-6bfb-4628-bb44-91087ba93096-kube-api-access-nvsd2\") pod \"monitoring-plugin-d5dcff7cf-rtj7l\" (UID: \"229ece0b-6bfb-4628-bb44-91087ba93096\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-rtj7l" Jul 06 23:50:20.060488 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:20.060412 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-rtj7l" Jul 06 23:50:20.071922 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:20.071892 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7c586d98b6-vr5mc"] Jul 06 23:50:20.094176 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:50:20.094143 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod376d341d_f1b9_4d45_90e8_4e77fe4f9837.slice/crio-d365a53e45b8a92238ee49f7f9d2ad59225422a96ba0826b62e409240c3f0e0e WatchSource:0}: Error finding container d365a53e45b8a92238ee49f7f9d2ad59225422a96ba0826b62e409240c3f0e0e: Status 404 returned error can't find the container with id d365a53e45b8a92238ee49f7f9d2ad59225422a96ba0826b62e409240c3f0e0e Jul 06 23:50:20.244735 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:20.244700 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kxl2w"] Jul 06 23:50:20.248640 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:50:20.248604 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b569175_860d_44bc_841d_a268294a112d.slice/crio-be044a5b5a916cfbf216f3e953c5a5dc0ea0c641833f4cbe7bbe08b0f59252a0 WatchSource:0}: Error finding container be044a5b5a916cfbf216f3e953c5a5dc0ea0c641833f4cbe7bbe08b0f59252a0: Status 404 returned error can't find the container with id be044a5b5a916cfbf216f3e953c5a5dc0ea0c641833f4cbe7bbe08b0f59252a0 Jul 06 23:50:20.263727 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:20.263701 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7666df4778-bvfdz"] Jul 06 23:50:20.265914 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:50:20.265885 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bb271dc_a651_471e_8d03_12423e38f85d.slice/crio-1b4119fa4eb865fe3f9b4503672aa0917d9dd9a6a0c16ac8649f7b6244de7c7f WatchSource:0}: Error finding container 1b4119fa4eb865fe3f9b4503672aa0917d9dd9a6a0c16ac8649f7b6244de7c7f: Status 404 returned error can't find the container with id 1b4119fa4eb865fe3f9b4503672aa0917d9dd9a6a0c16ac8649f7b6244de7c7f Jul 06 23:50:20.493502 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:20.493430 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-rtj7l"] Jul 06 23:50:20.496060 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:20.496017 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-f6cf5fb68-c5vnf"] Jul 06 23:50:20.496229 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:50:20.496016 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod229ece0b_6bfb_4628_bb44_91087ba93096.slice/crio-5f0e5225252f093332793bd935683f56c13195bafae64f4cc9197142e7740c2d WatchSource:0}: Error finding container 5f0e5225252f093332793bd935683f56c13195bafae64f4cc9197142e7740c2d: Status 404 returned error can't find the container with id 5f0e5225252f093332793bd935683f56c13195bafae64f4cc9197142e7740c2d Jul 06 23:50:20.499208 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:50:20.499185 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod503b39e8_97e2_4ad8_a4c5_0a716cd2061a.slice/crio-d271e44360c2732893ea32eef998e44e2cb75e6bda72115cc0eb304cee406076 WatchSource:0}: Error finding container d271e44360c2732893ea32eef998e44e2cb75e6bda72115cc0eb304cee406076: Status 404 returned error can't find the container with id d271e44360c2732893ea32eef998e44e2cb75e6bda72115cc0eb304cee406076 Jul 06 23:50:20.748436 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:20.748355 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qndcn" Jul 06 23:50:20.892285 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:20.892177 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7666df4778-bvfdz" event={"ID":"0bb271dc-a651-471e-8d03-12423e38f85d","Type":"ContainerStarted","Data":"3f258624188fd7fd061c3295e613bbc2493b71e84369876f3c3c3f5e75fe35af"} Jul 06 23:50:20.892285 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:20.892241 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7666df4778-bvfdz" event={"ID":"0bb271dc-a651-471e-8d03-12423e38f85d","Type":"ContainerStarted","Data":"1b4119fa4eb865fe3f9b4503672aa0917d9dd9a6a0c16ac8649f7b6244de7c7f"} Jul 06 23:50:20.894065 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:20.894031 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xg2fq" event={"ID":"376d341d-f1b9-4d45-90e8-4e77fe4f9837","Type":"ContainerStarted","Data":"d365a53e45b8a92238ee49f7f9d2ad59225422a96ba0826b62e409240c3f0e0e"} Jul 06 23:50:20.895736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:20.895701 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kxl2w" event={"ID":"4b569175-860d-44bc-841d-a268294a112d","Type":"ContainerStarted","Data":"be044a5b5a916cfbf216f3e953c5a5dc0ea0c641833f4cbe7bbe08b0f59252a0"} Jul 06 23:50:20.900821 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:20.900789 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-rtj7l" event={"ID":"229ece0b-6bfb-4628-bb44-91087ba93096","Type":"ContainerStarted","Data":"5f0e5225252f093332793bd935683f56c13195bafae64f4cc9197142e7740c2d"} Jul 06 23:50:20.903599 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:20.903570 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" event={"ID":"503b39e8-97e2-4ad8-a4c5-0a716cd2061a","Type":"ContainerStarted","Data":"d271e44360c2732893ea32eef998e44e2cb75e6bda72115cc0eb304cee406076"} Jul 06 23:50:20.917527 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:20.917459 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7666df4778-bvfdz" podStartSLOduration=1.9174424 podStartE2EDuration="1.9174424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:50:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:20.916039906 +0000 UTC m=+103.125769448" watchObservedRunningTime="2026-07-06 23:50:20.9174424 +0000 UTC m=+103.127171899" Jul 06 23:50:21.914093 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:21.914022 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-bvlqn" event={"ID":"a5c9b8dc-3d47-439f-89bf-01f9077f4043","Type":"ContainerStarted","Data":"8d4f13376db971d116327082ed90432c03ac6bb503fe6e437e9a47c0afc18aa4"} Jul 06 23:50:21.914975 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:21.914773 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-bvlqn" Jul 06 23:50:21.920099 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:21.920002 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xg2fq" event={"ID":"376d341d-f1b9-4d45-90e8-4e77fe4f9837","Type":"ContainerStarted","Data":"a200ad6895907820d35536d40222a7af8128786dc87aa8b40b567f31bf92f0ca"} Jul 06 23:50:21.928751 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:21.928692 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-bvlqn" Jul 06 23:50:21.935852 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:21.935426 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-bvlqn" podStartSLOduration=4.970482585 podStartE2EDuration="22.935406957s" totalImagesPullingTime="17.964924372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:21.932555775 +0000 UTC m=+104.142285302" watchObservedRunningTime="2026-07-06 23:50:21.935406957 +0000 UTC m=+104.145136478" Jul 06 23:50:22.928249 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:22.928211 2553 generic.go:356] "Generic (PLEG): container finished" podID="376d341d-f1b9-4d45-90e8-4e77fe4f9837" containerID="a200ad6895907820d35536d40222a7af8128786dc87aa8b40b567f31bf92f0ca" exitCode=0 Jul 06 23:50:22.929722 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:22.929551 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xg2fq" event={"ID":"376d341d-f1b9-4d45-90e8-4e77fe4f9837","Type":"ContainerDied","Data":"a200ad6895907820d35536d40222a7af8128786dc87aa8b40b567f31bf92f0ca"} Jul 06 23:50:24.941184 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:24.941143 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xg2fq" event={"ID":"376d341d-f1b9-4d45-90e8-4e77fe4f9837","Type":"ContainerStarted","Data":"bacb084dad7693a63742c861378781cef07ae2a8d62e6c1bbf3613dea4d87d37"} Jul 06 23:50:24.941655 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:24.941209 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xg2fq" event={"ID":"376d341d-f1b9-4d45-90e8-4e77fe4f9837","Type":"ContainerStarted","Data":"30b792b0a6141e6d9f49b6c89ffbe9a0a31c2232080a8320fec1aafa6103d60c"} Jul 06 23:50:24.943613 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:24.943579 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kxl2w" event={"ID":"4b569175-860d-44bc-841d-a268294a112d","Type":"ContainerStarted","Data":"90c42441997f3ab7921fcacdc25796f405be37bf85c3852547dd7a738ba9fe45"} Jul 06 23:50:24.943741 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:24.943639 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kxl2w" event={"ID":"4b569175-860d-44bc-841d-a268294a112d","Type":"ContainerStarted","Data":"ea21be97b2d1a2c10993bd3710814aedcf5b2d5605c5da180953d748aab6aeb3"} Jul 06 23:50:24.943741 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:24.943665 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kxl2w" event={"ID":"4b569175-860d-44bc-841d-a268294a112d","Type":"ContainerStarted","Data":"f810e0a9698df5b96a86e3ae44d3463df4dbc5a7a8cbcc54c172e680eccf52a7"} Jul 06 23:50:24.945595 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:24.945564 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-rtj7l" event={"ID":"229ece0b-6bfb-4628-bb44-91087ba93096","Type":"ContainerStarted","Data":"aebd8ebc20522371f5bcda25e99f01578faa9aa9bc8558f0943edc1b2f3bffa8"} Jul 06 23:50:24.945766 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:24.945746 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-rtj7l" Jul 06 23:50:24.947878 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:24.947470 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" event={"ID":"503b39e8-97e2-4ad8-a4c5-0a716cd2061a","Type":"ContainerStarted","Data":"a251209c78f401816d7ce4cf6c65509d63d2aeb08cc37b48c03335350ca6ef6b"} Jul 06 23:50:24.952417 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:24.952389 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-rtj7l" Jul 06 23:50:24.966150 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:24.966092 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xg2fq" podStartSLOduration=9.824172045 podStartE2EDuration="10.966064244s" totalImagesPullingTime="1.024734335s" totalInitContainerRuntime="117.157864ms" isStatefulPod=true podCreationTimestamp="2026-07-06 23:50:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:24.965960253 +0000 UTC m=+107.175689796" watchObservedRunningTime="2026-07-06 23:50:24.966064244 +0000 UTC m=+107.175793766" Jul 06 23:50:24.987658 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:24.987604 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-kxl2w" podStartSLOduration=7.3475401399999996 podStartE2EDuration="10.987587072s" totalImagesPullingTime="3.640046932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:50:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:24.986135178 +0000 UTC m=+107.195864699" watchObservedRunningTime="2026-07-06 23:50:24.987587072 +0000 UTC m=+107.197316595" Jul 06 23:50:25.005192 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:25.005135 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-rtj7l" podStartSLOduration=2.608877568 podStartE2EDuration="6.005119494s" totalImagesPullingTime="3.396241926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:50:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:25.003266055 +0000 UTC m=+107.212995587" watchObservedRunningTime="2026-07-06 23:50:25.005119494 +0000 UTC m=+107.214849016" Jul 06 23:50:25.027475 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:25.027423 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" podStartSLOduration=2.63125726 podStartE2EDuration="6.027402436s" totalImagesPullingTime="3.396145176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:50:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:25.025841279 +0000 UTC m=+107.235570802" watchObservedRunningTime="2026-07-06 23:50:25.027402436 +0000 UTC m=+107.237131957" Jul 06 23:50:29.048254 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:29.048211 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7666df4778-bvfdz"] Jul 06 23:50:29.487491 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:29.487448 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7666df4778-bvfdz" Jul 06 23:50:39.624457 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:39.624427 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" Jul 06 23:50:39.625020 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:39.624486 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" Jul 06 23:50:39.691817 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:39.691786 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bc4c4fdbb-9gqpg"] Jul 06 23:50:45.094257 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:45.094187 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" podUID="1bc2dc5e-4eab-40a3-9720-ff2ae2746cad" containerName="registry" containerID="cri-o://367d1741105f67257af66952de7e959bed07931970a9a56cf058e06f0d7b6802" gracePeriod=30 Jul 06 23:50:45.366784 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:45.366721 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:50:45.435548 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:45.435511 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-bound-sa-token\") pod \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " Jul 06 23:50:45.435548 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:45.435554 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-installation-pull-secrets\") pod \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " Jul 06 23:50:45.435777 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:45.435579 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-trusted-ca\") pod \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " Jul 06 23:50:45.435777 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:45.435610 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-tls\") pod \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " Jul 06 23:50:45.435777 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:45.435751 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jj4tb\" (UniqueName: \"kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-kube-api-access-jj4tb\") pod \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " Jul 06 23:50:45.435931 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:45.435809 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-ca-trust-extracted\") pod \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " Jul 06 23:50:45.435931 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:45.435881 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-certificates\") pod \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " Jul 06 23:50:45.435931 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:45.435917 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-image-registry-private-configuration\") pod \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\" (UID: \"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad\") " Jul 06 23:50:45.436192 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:45.436153 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "1bc2dc5e-4eab-40a3-9720-ff2ae2746cad" (UID: "1bc2dc5e-4eab-40a3-9720-ff2ae2746cad"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:50:45.436324 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:45.436273 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-trusted-ca\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:50:45.436571 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:45.436534 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "1bc2dc5e-4eab-40a3-9720-ff2ae2746cad" (UID: "1bc2dc5e-4eab-40a3-9720-ff2ae2746cad"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:50:45.438679 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:45.438652 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "1bc2dc5e-4eab-40a3-9720-ff2ae2746cad" (UID: "1bc2dc5e-4eab-40a3-9720-ff2ae2746cad"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:50:45.438799 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:45.438758 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-kube-api-access-jj4tb" (OuterVolumeSpecName: "kube-api-access-jj4tb") pod "1bc2dc5e-4eab-40a3-9720-ff2ae2746cad" (UID: "1bc2dc5e-4eab-40a3-9720-ff2ae2746cad"). InnerVolumeSpecName "kube-api-access-jj4tb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:50:45.438867 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:45.438794 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "1bc2dc5e-4eab-40a3-9720-ff2ae2746cad" (UID: "1bc2dc5e-4eab-40a3-9720-ff2ae2746cad"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:50:45.438867 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:45.438840 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "1bc2dc5e-4eab-40a3-9720-ff2ae2746cad" (UID: "1bc2dc5e-4eab-40a3-9720-ff2ae2746cad"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:50:45.438867 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:45.438849 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "1bc2dc5e-4eab-40a3-9720-ff2ae2746cad" (UID: "1bc2dc5e-4eab-40a3-9720-ff2ae2746cad"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:50:45.445198 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:45.445159 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "1bc2dc5e-4eab-40a3-9720-ff2ae2746cad" (UID: "1bc2dc5e-4eab-40a3-9720-ff2ae2746cad"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:50:45.537599 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:45.537560 2553 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-bound-sa-token\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:50:45.537599 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:45.537591 2553 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-installation-pull-secrets\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:50:45.537599 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:45.537602 2553 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-tls\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:50:45.537599 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:45.537611 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jj4tb\" (UniqueName: \"kubernetes.io/projected/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-kube-api-access-jj4tb\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:50:45.537864 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:45.537619 2553 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-ca-trust-extracted\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:50:45.537864 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:45.537629 2553 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-registry-certificates\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:50:45.537864 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:45.537640 2553 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad-image-registry-private-configuration\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:50:46.032797 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:46.032765 2553 generic.go:356] "Generic (PLEG): container finished" podID="1bc2dc5e-4eab-40a3-9720-ff2ae2746cad" containerID="367d1741105f67257af66952de7e959bed07931970a9a56cf058e06f0d7b6802" exitCode=0 Jul 06 23:50:46.032986 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:46.032811 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" event={"ID":"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad","Type":"ContainerDied","Data":"367d1741105f67257af66952de7e959bed07931970a9a56cf058e06f0d7b6802"} Jul 06 23:50:46.032986 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:46.032807 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" Jul 06 23:50:46.032986 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:46.032852 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c586d98b6-vr5mc" event={"ID":"1bc2dc5e-4eab-40a3-9720-ff2ae2746cad","Type":"ContainerDied","Data":"0aa641940ea12e8d755118a4af3c1fd40a948188c22838de9dddbbad5589c581"} Jul 06 23:50:46.032986 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:46.032873 2553 scope.go:117] "RemoveContainer" containerID="367d1741105f67257af66952de7e959bed07931970a9a56cf058e06f0d7b6802" Jul 06 23:50:46.042148 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:46.042128 2553 scope.go:117] "RemoveContainer" containerID="367d1741105f67257af66952de7e959bed07931970a9a56cf058e06f0d7b6802" Jul 06 23:50:46.042477 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:50:46.042454 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"367d1741105f67257af66952de7e959bed07931970a9a56cf058e06f0d7b6802\": container with ID starting with 367d1741105f67257af66952de7e959bed07931970a9a56cf058e06f0d7b6802 not found: ID does not exist" containerID="367d1741105f67257af66952de7e959bed07931970a9a56cf058e06f0d7b6802" Jul 06 23:50:46.042549 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:46.042485 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"367d1741105f67257af66952de7e959bed07931970a9a56cf058e06f0d7b6802"} err="failed to get container status \"367d1741105f67257af66952de7e959bed07931970a9a56cf058e06f0d7b6802\": rpc error: code = NotFound desc = could not find container \"367d1741105f67257af66952de7e959bed07931970a9a56cf058e06f0d7b6802\": container with ID starting with 367d1741105f67257af66952de7e959bed07931970a9a56cf058e06f0d7b6802 not found: ID does not exist" Jul 06 23:50:46.055370 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:46.055343 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7c586d98b6-vr5mc"] Jul 06 23:50:46.058683 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:46.058660 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7c586d98b6-vr5mc"] Jul 06 23:50:46.301124 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:46.301044 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bc2dc5e-4eab-40a3-9720-ff2ae2746cad" path="/var/lib/kubelet/pods/1bc2dc5e-4eab-40a3-9720-ff2ae2746cad/volumes" Jul 06 23:50:48.161714 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:48.161681 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/971187a6-5709-48db-b990-0a13db9fad13-metrics-certs\") pod \"network-metrics-daemon-zmkd8\" (UID: \"971187a6-5709-48db-b990-0a13db9fad13\") " pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:50:48.164069 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:48.164045 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/971187a6-5709-48db-b990-0a13db9fad13-metrics-certs\") pod \"network-metrics-daemon-zmkd8\" (UID: \"971187a6-5709-48db-b990-0a13db9fad13\") " pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:50:48.423468 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:48.423389 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nzg9j\"" Jul 06 23:50:48.430597 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:48.430571 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zmkd8" Jul 06 23:50:48.590950 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:48.590919 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zmkd8"] Jul 06 23:50:48.594045 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:50:48.594015 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod971187a6_5709_48db_b990_0a13db9fad13.slice/crio-b0ffa8ed8a7b8e283642970e956987198d338366677dc94eec5a4b3fd2eb1fea WatchSource:0}: Error finding container b0ffa8ed8a7b8e283642970e956987198d338366677dc94eec5a4b3fd2eb1fea: Status 404 returned error can't find the container with id b0ffa8ed8a7b8e283642970e956987198d338366677dc94eec5a4b3fd2eb1fea Jul 06 23:50:49.046018 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:49.045979 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zmkd8" event={"ID":"971187a6-5709-48db-b990-0a13db9fad13","Type":"ContainerStarted","Data":"b0ffa8ed8a7b8e283642970e956987198d338366677dc94eec5a4b3fd2eb1fea"} Jul 06 23:50:51.055236 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:51.055202 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zmkd8" event={"ID":"971187a6-5709-48db-b990-0a13db9fad13","Type":"ContainerStarted","Data":"4905c8f4c66e15c5db4e46314757e94ebb40348cf7e81a988c828e5e777689a8"} Jul 06 23:50:51.055644 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:51.055256 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zmkd8" event={"ID":"971187a6-5709-48db-b990-0a13db9fad13","Type":"ContainerStarted","Data":"832eb6ab46baa264231fa469a42a43ca45484d24c6a6a29b993e4264bf610b9a"} Jul 06 23:50:51.076866 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:51.076810 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zmkd8" podStartSLOduration=131.541053882 podStartE2EDuration="2m13.076792322s" totalImagesPullingTime="1.53573844s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:51.075455111 +0000 UTC m=+133.285184655" watchObservedRunningTime="2026-07-06 23:50:51.076792322 +0000 UTC m=+133.286521854" Jul 06 23:50:54.076665 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:54.076612 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7666df4778-bvfdz" podUID="0bb271dc-a651-471e-8d03-12423e38f85d" containerName="console" containerID="cri-o://3f258624188fd7fd061c3295e613bbc2493b71e84369876f3c3c3f5e75fe35af" gracePeriod=15 Jul 06 23:50:54.373587 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:54.373550 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7666df4778-bvfdz" Jul 06 23:50:54.517134 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:54.517094 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0bb271dc-a651-471e-8d03-12423e38f85d-console-oauth-config\") pod \"0bb271dc-a651-471e-8d03-12423e38f85d\" (UID: \"0bb271dc-a651-471e-8d03-12423e38f85d\") " Jul 06 23:50:54.517345 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:54.517145 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0bb271dc-a651-471e-8d03-12423e38f85d-trusted-ca-bundle\") pod \"0bb271dc-a651-471e-8d03-12423e38f85d\" (UID: \"0bb271dc-a651-471e-8d03-12423e38f85d\") " Jul 06 23:50:54.517345 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:54.517170 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0bb271dc-a651-471e-8d03-12423e38f85d-console-serving-cert\") pod \"0bb271dc-a651-471e-8d03-12423e38f85d\" (UID: \"0bb271dc-a651-471e-8d03-12423e38f85d\") " Jul 06 23:50:54.517345 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:54.517204 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0bb271dc-a651-471e-8d03-12423e38f85d-console-config\") pod \"0bb271dc-a651-471e-8d03-12423e38f85d\" (UID: \"0bb271dc-a651-471e-8d03-12423e38f85d\") " Jul 06 23:50:54.517345 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:54.517259 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0bb271dc-a651-471e-8d03-12423e38f85d-oauth-serving-cert\") pod \"0bb271dc-a651-471e-8d03-12423e38f85d\" (UID: \"0bb271dc-a651-471e-8d03-12423e38f85d\") " Jul 06 23:50:54.517345 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:54.517333 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfx5g\" (UniqueName: \"kubernetes.io/projected/0bb271dc-a651-471e-8d03-12423e38f85d-kube-api-access-hfx5g\") pod \"0bb271dc-a651-471e-8d03-12423e38f85d\" (UID: \"0bb271dc-a651-471e-8d03-12423e38f85d\") " Jul 06 23:50:54.517622 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:54.517373 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0bb271dc-a651-471e-8d03-12423e38f85d-service-ca\") pod \"0bb271dc-a651-471e-8d03-12423e38f85d\" (UID: \"0bb271dc-a651-471e-8d03-12423e38f85d\") " Jul 06 23:50:54.517726 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:54.517695 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bb271dc-a651-471e-8d03-12423e38f85d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0bb271dc-a651-471e-8d03-12423e38f85d" (UID: "0bb271dc-a651-471e-8d03-12423e38f85d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:50:54.517826 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:54.517716 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bb271dc-a651-471e-8d03-12423e38f85d-console-config" (OuterVolumeSpecName: "console-config") pod "0bb271dc-a651-471e-8d03-12423e38f85d" (UID: "0bb271dc-a651-471e-8d03-12423e38f85d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:50:54.517888 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:54.517807 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bb271dc-a651-471e-8d03-12423e38f85d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0bb271dc-a651-471e-8d03-12423e38f85d" (UID: "0bb271dc-a651-471e-8d03-12423e38f85d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:50:54.517888 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:54.517863 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bb271dc-a651-471e-8d03-12423e38f85d-service-ca" (OuterVolumeSpecName: "service-ca") pod "0bb271dc-a651-471e-8d03-12423e38f85d" (UID: "0bb271dc-a651-471e-8d03-12423e38f85d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:50:54.519625 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:54.519597 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bb271dc-a651-471e-8d03-12423e38f85d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0bb271dc-a651-471e-8d03-12423e38f85d" (UID: "0bb271dc-a651-471e-8d03-12423e38f85d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:50:54.519776 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:54.519753 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bb271dc-a651-471e-8d03-12423e38f85d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0bb271dc-a651-471e-8d03-12423e38f85d" (UID: "0bb271dc-a651-471e-8d03-12423e38f85d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:50:54.519829 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:54.519773 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bb271dc-a651-471e-8d03-12423e38f85d-kube-api-access-hfx5g" (OuterVolumeSpecName: "kube-api-access-hfx5g") pod "0bb271dc-a651-471e-8d03-12423e38f85d" (UID: "0bb271dc-a651-471e-8d03-12423e38f85d"). InnerVolumeSpecName "kube-api-access-hfx5g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:50:54.618656 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:54.618573 2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0bb271dc-a651-471e-8d03-12423e38f85d-oauth-serving-cert\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:50:54.618656 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:54.618606 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfx5g\" (UniqueName: \"kubernetes.io/projected/0bb271dc-a651-471e-8d03-12423e38f85d-kube-api-access-hfx5g\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:50:54.618656 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:54.618617 2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0bb271dc-a651-471e-8d03-12423e38f85d-service-ca\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:50:54.618656 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:54.618626 2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0bb271dc-a651-471e-8d03-12423e38f85d-console-oauth-config\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:50:54.618656 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:54.618634 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0bb271dc-a651-471e-8d03-12423e38f85d-trusted-ca-bundle\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:50:54.618656 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:54.618642 2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0bb271dc-a651-471e-8d03-12423e38f85d-console-serving-cert\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:50:54.618656 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:54.618650 2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0bb271dc-a651-471e-8d03-12423e38f85d-console-config\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:50:55.070504 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:55.070470 2553 generic.go:356] "Generic (PLEG): container finished" podID="0bb271dc-a651-471e-8d03-12423e38f85d" containerID="3f258624188fd7fd061c3295e613bbc2493b71e84369876f3c3c3f5e75fe35af" exitCode=0 Jul 06 23:50:55.070674 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:55.070519 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7666df4778-bvfdz" Jul 06 23:50:55.070674 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:55.070550 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7666df4778-bvfdz" event={"ID":"0bb271dc-a651-471e-8d03-12423e38f85d","Type":"ContainerDied","Data":"3f258624188fd7fd061c3295e613bbc2493b71e84369876f3c3c3f5e75fe35af"} Jul 06 23:50:55.070674 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:55.070604 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7666df4778-bvfdz" event={"ID":"0bb271dc-a651-471e-8d03-12423e38f85d","Type":"ContainerDied","Data":"1b4119fa4eb865fe3f9b4503672aa0917d9dd9a6a0c16ac8649f7b6244de7c7f"} Jul 06 23:50:55.070674 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:55.070628 2553 scope.go:117] "RemoveContainer" containerID="3f258624188fd7fd061c3295e613bbc2493b71e84369876f3c3c3f5e75fe35af" Jul 06 23:50:55.078969 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:55.078806 2553 scope.go:117] "RemoveContainer" containerID="3f258624188fd7fd061c3295e613bbc2493b71e84369876f3c3c3f5e75fe35af" Jul 06 23:50:55.079179 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:50:55.079073 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f258624188fd7fd061c3295e613bbc2493b71e84369876f3c3c3f5e75fe35af\": container with ID starting with 3f258624188fd7fd061c3295e613bbc2493b71e84369876f3c3c3f5e75fe35af not found: ID does not exist" containerID="3f258624188fd7fd061c3295e613bbc2493b71e84369876f3c3c3f5e75fe35af" Jul 06 23:50:55.079179 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:55.079096 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f258624188fd7fd061c3295e613bbc2493b71e84369876f3c3c3f5e75fe35af"} err="failed to get container status \"3f258624188fd7fd061c3295e613bbc2493b71e84369876f3c3c3f5e75fe35af\": rpc error: code = NotFound desc = could not find container \"3f258624188fd7fd061c3295e613bbc2493b71e84369876f3c3c3f5e75fe35af\": container with ID starting with 3f258624188fd7fd061c3295e613bbc2493b71e84369876f3c3c3f5e75fe35af not found: ID does not exist" Jul 06 23:50:55.090431 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:55.090410 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7666df4778-bvfdz"] Jul 06 23:50:55.096219 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:55.096197 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7666df4778-bvfdz"] Jul 06 23:50:56.302213 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:56.302182 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bb271dc-a651-471e-8d03-12423e38f85d" path="/var/lib/kubelet/pods/0bb271dc-a651-471e-8d03-12423e38f85d/volumes" Jul 06 23:50:59.629651 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:59.629619 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" Jul 06 23:50:59.633628 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:50:59.633601 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-f6cf5fb68-c5vnf" Jul 06 23:51:01.094911 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:01.094876 2553 generic.go:356] "Generic (PLEG): container finished" podID="5e219958-dd12-412f-920e-d732d6c4f806" containerID="6043fd5b7733b8dd9b0301f409b5b1c78ec80b4e0c158f7abe6853c41ebe259b" exitCode=0 Jul 06 23:51:01.095311 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:01.094950 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nks6v" event={"ID":"5e219958-dd12-412f-920e-d732d6c4f806","Type":"ContainerDied","Data":"6043fd5b7733b8dd9b0301f409b5b1c78ec80b4e0c158f7abe6853c41ebe259b"} Jul 06 23:51:01.095311 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:01.095267 2553 scope.go:117] "RemoveContainer" containerID="6043fd5b7733b8dd9b0301f409b5b1c78ec80b4e0c158f7abe6853c41ebe259b" Jul 06 23:51:02.100708 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:02.100675 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nks6v" event={"ID":"5e219958-dd12-412f-920e-d732d6c4f806","Type":"ContainerStarted","Data":"86d3dae5e2ff30972fa805045abc05b085faee72d972f137b6c7e66ecc1d7c61"} Jul 06 23:51:04.718108 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:04.718062 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-bc4c4fdbb-9gqpg" podUID="9a7e3b36-ef45-4488-bf49-e1dec85e7174" containerName="console" containerID="cri-o://311fe0546b9d957c52573d6c1674334162fb87892ec335f45245f962602f25a5" gracePeriod=15 Jul 06 23:51:05.027867 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:05.027824 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bc4c4fdbb-9gqpg" Jul 06 23:51:05.105254 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:05.105219 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a7e3b36-ef45-4488-bf49-e1dec85e7174-console-serving-cert\") pod \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\" (UID: \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\") " Jul 06 23:51:05.105458 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:05.105279 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a7e3b36-ef45-4488-bf49-e1dec85e7174-console-oauth-config\") pod \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\" (UID: \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\") " Jul 06 23:51:05.105458 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:05.105341 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a7e3b36-ef45-4488-bf49-e1dec85e7174-oauth-serving-cert\") pod \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\" (UID: \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\") " Jul 06 23:51:05.105458 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:05.105413 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a7e3b36-ef45-4488-bf49-e1dec85e7174-service-ca\") pod \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\" (UID: \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\") " Jul 06 23:51:05.105458 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:05.105448 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a7e3b36-ef45-4488-bf49-e1dec85e7174-console-config\") pod \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\" (UID: \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\") " Jul 06 23:51:05.105676 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:05.105474 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqqrt\" (UniqueName: \"kubernetes.io/projected/9a7e3b36-ef45-4488-bf49-e1dec85e7174-kube-api-access-hqqrt\") pod \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\" (UID: \"9a7e3b36-ef45-4488-bf49-e1dec85e7174\") " Jul 06 23:51:05.105748 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:05.105711 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a7e3b36-ef45-4488-bf49-e1dec85e7174-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9a7e3b36-ef45-4488-bf49-e1dec85e7174" (UID: "9a7e3b36-ef45-4488-bf49-e1dec85e7174"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:51:05.105972 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:05.105947 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a7e3b36-ef45-4488-bf49-e1dec85e7174-console-config" (OuterVolumeSpecName: "console-config") pod "9a7e3b36-ef45-4488-bf49-e1dec85e7174" (UID: "9a7e3b36-ef45-4488-bf49-e1dec85e7174"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:51:05.106116 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:05.106092 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a7e3b36-ef45-4488-bf49-e1dec85e7174-service-ca" (OuterVolumeSpecName: "service-ca") pod "9a7e3b36-ef45-4488-bf49-e1dec85e7174" (UID: "9a7e3b36-ef45-4488-bf49-e1dec85e7174"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:51:05.107514 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:05.107488 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a7e3b36-ef45-4488-bf49-e1dec85e7174-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9a7e3b36-ef45-4488-bf49-e1dec85e7174" (UID: "9a7e3b36-ef45-4488-bf49-e1dec85e7174"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:51:05.107625 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:05.107527 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a7e3b36-ef45-4488-bf49-e1dec85e7174-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9a7e3b36-ef45-4488-bf49-e1dec85e7174" (UID: "9a7e3b36-ef45-4488-bf49-e1dec85e7174"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:51:05.107625 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:05.107546 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a7e3b36-ef45-4488-bf49-e1dec85e7174-kube-api-access-hqqrt" (OuterVolumeSpecName: "kube-api-access-hqqrt") pod "9a7e3b36-ef45-4488-bf49-e1dec85e7174" (UID: "9a7e3b36-ef45-4488-bf49-e1dec85e7174"). InnerVolumeSpecName "kube-api-access-hqqrt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:51:05.119763 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:05.115723 2553 generic.go:356] "Generic (PLEG): container finished" podID="9a7e3b36-ef45-4488-bf49-e1dec85e7174" containerID="311fe0546b9d957c52573d6c1674334162fb87892ec335f45245f962602f25a5" exitCode=0 Jul 06 23:51:05.119763 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:05.115781 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bc4c4fdbb-9gqpg" event={"ID":"9a7e3b36-ef45-4488-bf49-e1dec85e7174","Type":"ContainerDied","Data":"311fe0546b9d957c52573d6c1674334162fb87892ec335f45245f962602f25a5"} Jul 06 23:51:05.119763 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:05.115827 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bc4c4fdbb-9gqpg" event={"ID":"9a7e3b36-ef45-4488-bf49-e1dec85e7174","Type":"ContainerDied","Data":"86a10f4e5b976e4113475dc7813fc6fcc730c6a3ebb34eb2d808a3fe5d64e416"} Jul 06 23:51:05.119763 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:05.115852 2553 scope.go:117] "RemoveContainer" containerID="311fe0546b9d957c52573d6c1674334162fb87892ec335f45245f962602f25a5" Jul 06 23:51:05.119763 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:05.116023 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bc4c4fdbb-9gqpg" Jul 06 23:51:05.134922 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:05.134900 2553 scope.go:117] "RemoveContainer" containerID="311fe0546b9d957c52573d6c1674334162fb87892ec335f45245f962602f25a5" Jul 06 23:51:05.135363 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:51:05.135327 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"311fe0546b9d957c52573d6c1674334162fb87892ec335f45245f962602f25a5\": container with ID starting with 311fe0546b9d957c52573d6c1674334162fb87892ec335f45245f962602f25a5 not found: ID does not exist" containerID="311fe0546b9d957c52573d6c1674334162fb87892ec335f45245f962602f25a5" Jul 06 23:51:05.135474 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:05.135378 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"311fe0546b9d957c52573d6c1674334162fb87892ec335f45245f962602f25a5"} err="failed to get container status \"311fe0546b9d957c52573d6c1674334162fb87892ec335f45245f962602f25a5\": rpc error: code = NotFound desc = could not find container \"311fe0546b9d957c52573d6c1674334162fb87892ec335f45245f962602f25a5\": container with ID starting with 311fe0546b9d957c52573d6c1674334162fb87892ec335f45245f962602f25a5 not found: ID does not exist" Jul 06 23:51:05.143575 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:05.143545 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bc4c4fdbb-9gqpg"] Jul 06 23:51:05.148082 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:05.148054 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-bc4c4fdbb-9gqpg"] Jul 06 23:51:05.206340 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:05.206285 2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a7e3b36-ef45-4488-bf49-e1dec85e7174-console-oauth-config\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:51:05.206340 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:05.206331 2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a7e3b36-ef45-4488-bf49-e1dec85e7174-oauth-serving-cert\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:51:05.206340 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:05.206344 2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a7e3b36-ef45-4488-bf49-e1dec85e7174-service-ca\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:51:05.206340 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:05.206352 2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a7e3b36-ef45-4488-bf49-e1dec85e7174-console-config\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:51:05.206614 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:05.206361 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqqrt\" (UniqueName: \"kubernetes.io/projected/9a7e3b36-ef45-4488-bf49-e1dec85e7174-kube-api-access-hqqrt\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:51:05.206614 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:05.206369 2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a7e3b36-ef45-4488-bf49-e1dec85e7174-console-serving-cert\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:51:06.301440 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:06.301406 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a7e3b36-ef45-4488-bf49-e1dec85e7174" path="/var/lib/kubelet/pods/9a7e3b36-ef45-4488-bf49-e1dec85e7174/volumes" Jul 06 23:51:43.649623 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.649584 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-676b998b6d-ct9fd"] Jul 06 23:51:43.650214 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.650162 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bb271dc-a651-471e-8d03-12423e38f85d" containerName="console" Jul 06 23:51:43.650214 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.650188 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bb271dc-a651-471e-8d03-12423e38f85d" containerName="console" Jul 06 23:51:43.650214 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.650214 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bc2dc5e-4eab-40a3-9720-ff2ae2746cad" containerName="registry" Jul 06 23:51:43.650423 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.650223 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bc2dc5e-4eab-40a3-9720-ff2ae2746cad" containerName="registry" Jul 06 23:51:43.650423 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.650233 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a7e3b36-ef45-4488-bf49-e1dec85e7174" containerName="console" Jul 06 23:51:43.650423 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.650242 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a7e3b36-ef45-4488-bf49-e1dec85e7174" containerName="console" Jul 06 23:51:43.650423 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.650346 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bc2dc5e-4eab-40a3-9720-ff2ae2746cad" containerName="registry" Jul 06 23:51:43.650423 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.650362 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a7e3b36-ef45-4488-bf49-e1dec85e7174" containerName="console" Jul 06 23:51:43.650423 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.650370 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bb271dc-a651-471e-8d03-12423e38f85d" containerName="console" Jul 06 23:51:43.654720 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.654697 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-676b998b6d-ct9fd" Jul 06 23:51:43.659252 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.659222 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 23:51:43.659435 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.659313 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5bz9s\"" Jul 06 23:51:43.659435 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.659328 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 23:51:43.659435 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.659315 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 23:51:43.659622 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.659485 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 23:51:43.659622 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.659491 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 23:51:43.665237 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.665210 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 23:51:43.666698 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.666672 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-676b998b6d-ct9fd"] Jul 06 23:51:43.796117 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.796056 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a45a3275-3d71-4c1b-9a17-82174da36004-service-ca\") pod \"console-676b998b6d-ct9fd\" (UID: \"a45a3275-3d71-4c1b-9a17-82174da36004\") " pod="openshift-console/console-676b998b6d-ct9fd" Jul 06 23:51:43.796117 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.796119 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a45a3275-3d71-4c1b-9a17-82174da36004-console-serving-cert\") pod \"console-676b998b6d-ct9fd\" (UID: \"a45a3275-3d71-4c1b-9a17-82174da36004\") " pod="openshift-console/console-676b998b6d-ct9fd" Jul 06 23:51:43.796419 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.796152 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a45a3275-3d71-4c1b-9a17-82174da36004-oauth-serving-cert\") pod \"console-676b998b6d-ct9fd\" (UID: \"a45a3275-3d71-4c1b-9a17-82174da36004\") " pod="openshift-console/console-676b998b6d-ct9fd" Jul 06 23:51:43.796419 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.796188 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pqxl\" (UniqueName: \"kubernetes.io/projected/a45a3275-3d71-4c1b-9a17-82174da36004-kube-api-access-8pqxl\") pod \"console-676b998b6d-ct9fd\" (UID: \"a45a3275-3d71-4c1b-9a17-82174da36004\") " pod="openshift-console/console-676b998b6d-ct9fd" Jul 06 23:51:43.796419 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.796280 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a45a3275-3d71-4c1b-9a17-82174da36004-console-oauth-config\") pod \"console-676b998b6d-ct9fd\" (UID: \"a45a3275-3d71-4c1b-9a17-82174da36004\") " pod="openshift-console/console-676b998b6d-ct9fd" Jul 06 23:51:43.796419 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.796362 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a45a3275-3d71-4c1b-9a17-82174da36004-trusted-ca-bundle\") pod \"console-676b998b6d-ct9fd\" (UID: \"a45a3275-3d71-4c1b-9a17-82174da36004\") " pod="openshift-console/console-676b998b6d-ct9fd" Jul 06 23:51:43.796419 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.796384 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a45a3275-3d71-4c1b-9a17-82174da36004-console-config\") pod \"console-676b998b6d-ct9fd\" (UID: \"a45a3275-3d71-4c1b-9a17-82174da36004\") " pod="openshift-console/console-676b998b6d-ct9fd" Jul 06 23:51:43.897278 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.897232 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a45a3275-3d71-4c1b-9a17-82174da36004-service-ca\") pod \"console-676b998b6d-ct9fd\" (UID: \"a45a3275-3d71-4c1b-9a17-82174da36004\") " pod="openshift-console/console-676b998b6d-ct9fd" Jul 06 23:51:43.897278 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.897274 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a45a3275-3d71-4c1b-9a17-82174da36004-console-serving-cert\") pod \"console-676b998b6d-ct9fd\" (UID: \"a45a3275-3d71-4c1b-9a17-82174da36004\") " pod="openshift-console/console-676b998b6d-ct9fd" Jul 06 23:51:43.897605 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.897290 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a45a3275-3d71-4c1b-9a17-82174da36004-oauth-serving-cert\") pod \"console-676b998b6d-ct9fd\" (UID: \"a45a3275-3d71-4c1b-9a17-82174da36004\") " pod="openshift-console/console-676b998b6d-ct9fd" Jul 06 23:51:43.897605 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.897355 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pqxl\" (UniqueName: \"kubernetes.io/projected/a45a3275-3d71-4c1b-9a17-82174da36004-kube-api-access-8pqxl\") pod \"console-676b998b6d-ct9fd\" (UID: \"a45a3275-3d71-4c1b-9a17-82174da36004\") " pod="openshift-console/console-676b998b6d-ct9fd" Jul 06 23:51:43.897605 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.897410 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a45a3275-3d71-4c1b-9a17-82174da36004-console-oauth-config\") pod \"console-676b998b6d-ct9fd\" (UID: \"a45a3275-3d71-4c1b-9a17-82174da36004\") " pod="openshift-console/console-676b998b6d-ct9fd" Jul 06 23:51:43.897605 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.897455 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a45a3275-3d71-4c1b-9a17-82174da36004-trusted-ca-bundle\") pod \"console-676b998b6d-ct9fd\" (UID: \"a45a3275-3d71-4c1b-9a17-82174da36004\") " pod="openshift-console/console-676b998b6d-ct9fd" Jul 06 23:51:43.897605 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.897474 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a45a3275-3d71-4c1b-9a17-82174da36004-console-config\") pod \"console-676b998b6d-ct9fd\" (UID: \"a45a3275-3d71-4c1b-9a17-82174da36004\") " pod="openshift-console/console-676b998b6d-ct9fd" Jul 06 23:51:43.898098 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.898075 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a45a3275-3d71-4c1b-9a17-82174da36004-service-ca\") pod \"console-676b998b6d-ct9fd\" (UID: \"a45a3275-3d71-4c1b-9a17-82174da36004\") " pod="openshift-console/console-676b998b6d-ct9fd" Jul 06 23:51:43.898192 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.898159 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a45a3275-3d71-4c1b-9a17-82174da36004-oauth-serving-cert\") pod \"console-676b998b6d-ct9fd\" (UID: \"a45a3275-3d71-4c1b-9a17-82174da36004\") " pod="openshift-console/console-676b998b6d-ct9fd" Jul 06 23:51:43.898192 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.898171 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a45a3275-3d71-4c1b-9a17-82174da36004-console-config\") pod \"console-676b998b6d-ct9fd\" (UID: \"a45a3275-3d71-4c1b-9a17-82174da36004\") " pod="openshift-console/console-676b998b6d-ct9fd" Jul 06 23:51:43.898325 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.898287 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a45a3275-3d71-4c1b-9a17-82174da36004-trusted-ca-bundle\") pod \"console-676b998b6d-ct9fd\" (UID: \"a45a3275-3d71-4c1b-9a17-82174da36004\") " pod="openshift-console/console-676b998b6d-ct9fd" Jul 06 23:51:43.899871 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.899817 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a45a3275-3d71-4c1b-9a17-82174da36004-console-oauth-config\") pod \"console-676b998b6d-ct9fd\" (UID: \"a45a3275-3d71-4c1b-9a17-82174da36004\") " pod="openshift-console/console-676b998b6d-ct9fd" Jul 06 23:51:43.900018 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.899999 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a45a3275-3d71-4c1b-9a17-82174da36004-console-serving-cert\") pod \"console-676b998b6d-ct9fd\" (UID: \"a45a3275-3d71-4c1b-9a17-82174da36004\") " pod="openshift-console/console-676b998b6d-ct9fd" Jul 06 23:51:43.906606 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.906571 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pqxl\" (UniqueName: \"kubernetes.io/projected/a45a3275-3d71-4c1b-9a17-82174da36004-kube-api-access-8pqxl\") pod \"console-676b998b6d-ct9fd\" (UID: \"a45a3275-3d71-4c1b-9a17-82174da36004\") " pod="openshift-console/console-676b998b6d-ct9fd" Jul 06 23:51:43.967499 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:43.967458 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-676b998b6d-ct9fd" Jul 06 23:51:44.097924 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:44.097891 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-676b998b6d-ct9fd"] Jul 06 23:51:44.101030 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:51:44.101003 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda45a3275_3d71_4c1b_9a17_82174da36004.slice/crio-12663aa10705e0847c2eee7cc364187b8fa062c01f1e74d52b285aaf00127cf7 WatchSource:0}: Error finding container 12663aa10705e0847c2eee7cc364187b8fa062c01f1e74d52b285aaf00127cf7: Status 404 returned error can't find the container with id 12663aa10705e0847c2eee7cc364187b8fa062c01f1e74d52b285aaf00127cf7 Jul 06 23:51:44.266016 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:44.265979 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-676b998b6d-ct9fd" event={"ID":"a45a3275-3d71-4c1b-9a17-82174da36004","Type":"ContainerStarted","Data":"f9d45bf4550d91de919e1b2bae371c51e28c4c1f7695b6a9b77af80b3b4973f2"} Jul 06 23:51:44.266199 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:44.266038 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-676b998b6d-ct9fd" event={"ID":"a45a3275-3d71-4c1b-9a17-82174da36004","Type":"ContainerStarted","Data":"12663aa10705e0847c2eee7cc364187b8fa062c01f1e74d52b285aaf00127cf7"} Jul 06 23:51:53.968571 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:53.968514 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-676b998b6d-ct9fd" Jul 06 23:51:53.969016 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:53.968596 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-676b998b6d-ct9fd" Jul 06 23:51:53.973448 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:53.973423 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-676b998b6d-ct9fd" Jul 06 23:51:53.994430 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:53.994372 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-676b998b6d-ct9fd" podStartSLOduration=10.994354885 podStartE2EDuration="10.994354885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:51:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:51:44.283350344 +0000 UTC m=+186.493079884" watchObservedRunningTime="2026-07-06 23:51:53.994354885 +0000 UTC m=+196.204084406" Jul 06 23:51:54.310944 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:51:54.310859 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-676b998b6d-ct9fd" Jul 06 23:52:59.546452 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:52:59.546415 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7ddd55fbb6-94gf2"] Jul 06 23:52:59.550542 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:52:59.550517 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ddd55fbb6-94gf2" Jul 06 23:52:59.561406 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:52:59.561382 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7ddd55fbb6-94gf2"] Jul 06 23:52:59.578583 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:52:59.578545 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f108968-7328-4771-92b3-5db9e1a812dc-oauth-serving-cert\") pod \"console-7ddd55fbb6-94gf2\" (UID: \"4f108968-7328-4771-92b3-5db9e1a812dc\") " pod="openshift-console/console-7ddd55fbb6-94gf2" Jul 06 23:52:59.578753 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:52:59.578592 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f108968-7328-4771-92b3-5db9e1a812dc-console-serving-cert\") pod \"console-7ddd55fbb6-94gf2\" (UID: \"4f108968-7328-4771-92b3-5db9e1a812dc\") " pod="openshift-console/console-7ddd55fbb6-94gf2" Jul 06 23:52:59.578753 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:52:59.578629 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f108968-7328-4771-92b3-5db9e1a812dc-service-ca\") pod \"console-7ddd55fbb6-94gf2\" (UID: \"4f108968-7328-4771-92b3-5db9e1a812dc\") " pod="openshift-console/console-7ddd55fbb6-94gf2" Jul 06 23:52:59.578753 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:52:59.578658 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f108968-7328-4771-92b3-5db9e1a812dc-console-oauth-config\") pod \"console-7ddd55fbb6-94gf2\" (UID: \"4f108968-7328-4771-92b3-5db9e1a812dc\") " pod="openshift-console/console-7ddd55fbb6-94gf2" Jul 06 23:52:59.578753 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:52:59.578679 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kn4fx\" (UniqueName: \"kubernetes.io/projected/4f108968-7328-4771-92b3-5db9e1a812dc-kube-api-access-kn4fx\") pod \"console-7ddd55fbb6-94gf2\" (UID: \"4f108968-7328-4771-92b3-5db9e1a812dc\") " pod="openshift-console/console-7ddd55fbb6-94gf2" Jul 06 23:52:59.578753 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:52:59.578704 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f108968-7328-4771-92b3-5db9e1a812dc-trusted-ca-bundle\") pod \"console-7ddd55fbb6-94gf2\" (UID: \"4f108968-7328-4771-92b3-5db9e1a812dc\") " pod="openshift-console/console-7ddd55fbb6-94gf2" Jul 06 23:52:59.578753 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:52:59.578720 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f108968-7328-4771-92b3-5db9e1a812dc-console-config\") pod \"console-7ddd55fbb6-94gf2\" (UID: \"4f108968-7328-4771-92b3-5db9e1a812dc\") " pod="openshift-console/console-7ddd55fbb6-94gf2" Jul 06 23:52:59.679945 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:52:59.679908 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f108968-7328-4771-92b3-5db9e1a812dc-console-config\") pod \"console-7ddd55fbb6-94gf2\" (UID: \"4f108968-7328-4771-92b3-5db9e1a812dc\") " pod="openshift-console/console-7ddd55fbb6-94gf2" Jul 06 23:52:59.679945 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:52:59.679958 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f108968-7328-4771-92b3-5db9e1a812dc-oauth-serving-cert\") pod \"console-7ddd55fbb6-94gf2\" (UID: \"4f108968-7328-4771-92b3-5db9e1a812dc\") " pod="openshift-console/console-7ddd55fbb6-94gf2" Jul 06 23:52:59.680195 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:52:59.680031 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f108968-7328-4771-92b3-5db9e1a812dc-console-serving-cert\") pod \"console-7ddd55fbb6-94gf2\" (UID: \"4f108968-7328-4771-92b3-5db9e1a812dc\") " pod="openshift-console/console-7ddd55fbb6-94gf2" Jul 06 23:52:59.680195 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:52:59.680108 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f108968-7328-4771-92b3-5db9e1a812dc-service-ca\") pod \"console-7ddd55fbb6-94gf2\" (UID: \"4f108968-7328-4771-92b3-5db9e1a812dc\") " pod="openshift-console/console-7ddd55fbb6-94gf2" Jul 06 23:52:59.680195 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:52:59.680168 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f108968-7328-4771-92b3-5db9e1a812dc-console-oauth-config\") pod \"console-7ddd55fbb6-94gf2\" (UID: \"4f108968-7328-4771-92b3-5db9e1a812dc\") " pod="openshift-console/console-7ddd55fbb6-94gf2" Jul 06 23:52:59.680195 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:52:59.680193 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kn4fx\" (UniqueName: \"kubernetes.io/projected/4f108968-7328-4771-92b3-5db9e1a812dc-kube-api-access-kn4fx\") pod \"console-7ddd55fbb6-94gf2\" (UID: \"4f108968-7328-4771-92b3-5db9e1a812dc\") " pod="openshift-console/console-7ddd55fbb6-94gf2" Jul 06 23:52:59.680437 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:52:59.680226 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f108968-7328-4771-92b3-5db9e1a812dc-trusted-ca-bundle\") pod \"console-7ddd55fbb6-94gf2\" (UID: \"4f108968-7328-4771-92b3-5db9e1a812dc\") " pod="openshift-console/console-7ddd55fbb6-94gf2" Jul 06 23:52:59.680744 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:52:59.680718 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f108968-7328-4771-92b3-5db9e1a812dc-oauth-serving-cert\") pod \"console-7ddd55fbb6-94gf2\" (UID: \"4f108968-7328-4771-92b3-5db9e1a812dc\") " pod="openshift-console/console-7ddd55fbb6-94gf2" Jul 06 23:52:59.680862 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:52:59.680779 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f108968-7328-4771-92b3-5db9e1a812dc-console-config\") pod \"console-7ddd55fbb6-94gf2\" (UID: \"4f108968-7328-4771-92b3-5db9e1a812dc\") " pod="openshift-console/console-7ddd55fbb6-94gf2" Jul 06 23:52:59.680904 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:52:59.680874 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f108968-7328-4771-92b3-5db9e1a812dc-service-ca\") pod \"console-7ddd55fbb6-94gf2\" (UID: \"4f108968-7328-4771-92b3-5db9e1a812dc\") " pod="openshift-console/console-7ddd55fbb6-94gf2" Jul 06 23:52:59.681150 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:52:59.681132 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f108968-7328-4771-92b3-5db9e1a812dc-trusted-ca-bundle\") pod \"console-7ddd55fbb6-94gf2\" (UID: \"4f108968-7328-4771-92b3-5db9e1a812dc\") " pod="openshift-console/console-7ddd55fbb6-94gf2" Jul 06 23:52:59.682732 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:52:59.682698 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f108968-7328-4771-92b3-5db9e1a812dc-console-serving-cert\") pod \"console-7ddd55fbb6-94gf2\" (UID: \"4f108968-7328-4771-92b3-5db9e1a812dc\") " pod="openshift-console/console-7ddd55fbb6-94gf2" Jul 06 23:52:59.682830 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:52:59.682709 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f108968-7328-4771-92b3-5db9e1a812dc-console-oauth-config\") pod \"console-7ddd55fbb6-94gf2\" (UID: \"4f108968-7328-4771-92b3-5db9e1a812dc\") " pod="openshift-console/console-7ddd55fbb6-94gf2" Jul 06 23:52:59.688257 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:52:59.688235 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kn4fx\" (UniqueName: \"kubernetes.io/projected/4f108968-7328-4771-92b3-5db9e1a812dc-kube-api-access-kn4fx\") pod \"console-7ddd55fbb6-94gf2\" (UID: \"4f108968-7328-4771-92b3-5db9e1a812dc\") " pod="openshift-console/console-7ddd55fbb6-94gf2" Jul 06 23:52:59.859850 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:52:59.859742 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ddd55fbb6-94gf2" Jul 06 23:52:59.994495 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:52:59.994467 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7ddd55fbb6-94gf2"] Jul 06 23:52:59.997174 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:52:59.997132 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f108968_7328_4771_92b3_5db9e1a812dc.slice/crio-360fb2c24d9579b76d523027b450200a98af6b523ce5b8965111bf3558370f5f WatchSource:0}: Error finding container 360fb2c24d9579b76d523027b450200a98af6b523ce5b8965111bf3558370f5f: Status 404 returned error can't find the container with id 360fb2c24d9579b76d523027b450200a98af6b523ce5b8965111bf3558370f5f Jul 06 23:53:00.546278 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:00.546242 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ddd55fbb6-94gf2" event={"ID":"4f108968-7328-4771-92b3-5db9e1a812dc","Type":"ContainerStarted","Data":"75c6506bd9e7916874249ff6e6b348fa8f10f302c95fc4497d5ce2904606508c"} Jul 06 23:53:00.546467 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:00.546316 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ddd55fbb6-94gf2" event={"ID":"4f108968-7328-4771-92b3-5db9e1a812dc","Type":"ContainerStarted","Data":"360fb2c24d9579b76d523027b450200a98af6b523ce5b8965111bf3558370f5f"} Jul 06 23:53:00.565245 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:00.565187 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7ddd55fbb6-94gf2" podStartSLOduration=1.565170349 podStartE2EDuration="1.565170349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:52:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:53:00.564542289 +0000 UTC m=+262.774271831" watchObservedRunningTime="2026-07-06 23:53:00.565170349 +0000 UTC m=+262.774899928" Jul 06 23:53:09.860411 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:09.860372 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7ddd55fbb6-94gf2" Jul 06 23:53:09.860803 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:09.860456 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7ddd55fbb6-94gf2" Jul 06 23:53:09.865218 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:09.865196 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7ddd55fbb6-94gf2" Jul 06 23:53:10.584871 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:10.584844 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7ddd55fbb6-94gf2" Jul 06 23:53:10.649514 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:10.649474 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-676b998b6d-ct9fd"] Jul 06 23:53:35.670315 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:35.670221 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-676b998b6d-ct9fd" podUID="a45a3275-3d71-4c1b-9a17-82174da36004" containerName="console" containerID="cri-o://f9d45bf4550d91de919e1b2bae371c51e28c4c1f7695b6a9b77af80b3b4973f2" gracePeriod=15 Jul 06 23:53:35.912603 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:35.912575 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-676b998b6d-ct9fd" Jul 06 23:53:36.082471 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.082364 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pqxl\" (UniqueName: \"kubernetes.io/projected/a45a3275-3d71-4c1b-9a17-82174da36004-kube-api-access-8pqxl\") pod \"a45a3275-3d71-4c1b-9a17-82174da36004\" (UID: \"a45a3275-3d71-4c1b-9a17-82174da36004\") " Jul 06 23:53:36.082471 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.082421 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a45a3275-3d71-4c1b-9a17-82174da36004-oauth-serving-cert\") pod \"a45a3275-3d71-4c1b-9a17-82174da36004\" (UID: \"a45a3275-3d71-4c1b-9a17-82174da36004\") " Jul 06 23:53:36.082471 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.082464 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a45a3275-3d71-4c1b-9a17-82174da36004-console-oauth-config\") pod \"a45a3275-3d71-4c1b-9a17-82174da36004\" (UID: \"a45a3275-3d71-4c1b-9a17-82174da36004\") " Jul 06 23:53:36.082747 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.082490 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a45a3275-3d71-4c1b-9a17-82174da36004-console-config\") pod \"a45a3275-3d71-4c1b-9a17-82174da36004\" (UID: \"a45a3275-3d71-4c1b-9a17-82174da36004\") " Jul 06 23:53:36.082747 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.082530 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a45a3275-3d71-4c1b-9a17-82174da36004-trusted-ca-bundle\") pod \"a45a3275-3d71-4c1b-9a17-82174da36004\" (UID: \"a45a3275-3d71-4c1b-9a17-82174da36004\") " Jul 06 23:53:36.082747 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.082560 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a45a3275-3d71-4c1b-9a17-82174da36004-console-serving-cert\") pod \"a45a3275-3d71-4c1b-9a17-82174da36004\" (UID: \"a45a3275-3d71-4c1b-9a17-82174da36004\") " Jul 06 23:53:36.082747 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.082603 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a45a3275-3d71-4c1b-9a17-82174da36004-service-ca\") pod \"a45a3275-3d71-4c1b-9a17-82174da36004\" (UID: \"a45a3275-3d71-4c1b-9a17-82174da36004\") " Jul 06 23:53:36.082963 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.082932 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a45a3275-3d71-4c1b-9a17-82174da36004-console-config" (OuterVolumeSpecName: "console-config") pod "a45a3275-3d71-4c1b-9a17-82174da36004" (UID: "a45a3275-3d71-4c1b-9a17-82174da36004"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:53:36.083015 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.082975 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a45a3275-3d71-4c1b-9a17-82174da36004-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a45a3275-3d71-4c1b-9a17-82174da36004" (UID: "a45a3275-3d71-4c1b-9a17-82174da36004"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:53:36.083083 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.082941 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a45a3275-3d71-4c1b-9a17-82174da36004-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a45a3275-3d71-4c1b-9a17-82174da36004" (UID: "a45a3275-3d71-4c1b-9a17-82174da36004"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:53:36.083169 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.083150 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a45a3275-3d71-4c1b-9a17-82174da36004-service-ca" (OuterVolumeSpecName: "service-ca") pod "a45a3275-3d71-4c1b-9a17-82174da36004" (UID: "a45a3275-3d71-4c1b-9a17-82174da36004"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:53:36.084637 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.084615 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a45a3275-3d71-4c1b-9a17-82174da36004-kube-api-access-8pqxl" (OuterVolumeSpecName: "kube-api-access-8pqxl") pod "a45a3275-3d71-4c1b-9a17-82174da36004" (UID: "a45a3275-3d71-4c1b-9a17-82174da36004"). InnerVolumeSpecName "kube-api-access-8pqxl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:53:36.085109 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.085078 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a45a3275-3d71-4c1b-9a17-82174da36004-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a45a3275-3d71-4c1b-9a17-82174da36004" (UID: "a45a3275-3d71-4c1b-9a17-82174da36004"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:53:36.085109 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.085092 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a45a3275-3d71-4c1b-9a17-82174da36004-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a45a3275-3d71-4c1b-9a17-82174da36004" (UID: "a45a3275-3d71-4c1b-9a17-82174da36004"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:53:36.183917 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.183872 2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a45a3275-3d71-4c1b-9a17-82174da36004-oauth-serving-cert\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:53:36.183917 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.183905 2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a45a3275-3d71-4c1b-9a17-82174da36004-console-oauth-config\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:53:36.183917 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.183915 2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a45a3275-3d71-4c1b-9a17-82174da36004-console-config\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:53:36.183917 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.183926 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a45a3275-3d71-4c1b-9a17-82174da36004-trusted-ca-bundle\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:53:36.184203 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.183936 2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a45a3275-3d71-4c1b-9a17-82174da36004-console-serving-cert\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:53:36.184203 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.183945 2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a45a3275-3d71-4c1b-9a17-82174da36004-service-ca\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:53:36.184203 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.183953 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pqxl\" (UniqueName: \"kubernetes.io/projected/a45a3275-3d71-4c1b-9a17-82174da36004-kube-api-access-8pqxl\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:53:36.677358 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.677288 2553 generic.go:356] "Generic (PLEG): container finished" podID="a45a3275-3d71-4c1b-9a17-82174da36004" containerID="f9d45bf4550d91de919e1b2bae371c51e28c4c1f7695b6a9b77af80b3b4973f2" exitCode=0 Jul 06 23:53:36.677358 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.677325 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-676b998b6d-ct9fd" event={"ID":"a45a3275-3d71-4c1b-9a17-82174da36004","Type":"ContainerDied","Data":"f9d45bf4550d91de919e1b2bae371c51e28c4c1f7695b6a9b77af80b3b4973f2"} Jul 06 23:53:36.677835 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.677372 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-676b998b6d-ct9fd" Jul 06 23:53:36.677835 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.677385 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-676b998b6d-ct9fd" event={"ID":"a45a3275-3d71-4c1b-9a17-82174da36004","Type":"ContainerDied","Data":"12663aa10705e0847c2eee7cc364187b8fa062c01f1e74d52b285aaf00127cf7"} Jul 06 23:53:36.677835 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.677407 2553 scope.go:117] "RemoveContainer" containerID="f9d45bf4550d91de919e1b2bae371c51e28c4c1f7695b6a9b77af80b3b4973f2" Jul 06 23:53:36.685714 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.685687 2553 scope.go:117] "RemoveContainer" containerID="f9d45bf4550d91de919e1b2bae371c51e28c4c1f7695b6a9b77af80b3b4973f2" Jul 06 23:53:36.686000 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:53:36.685982 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9d45bf4550d91de919e1b2bae371c51e28c4c1f7695b6a9b77af80b3b4973f2\": container with ID starting with f9d45bf4550d91de919e1b2bae371c51e28c4c1f7695b6a9b77af80b3b4973f2 not found: ID does not exist" containerID="f9d45bf4550d91de919e1b2bae371c51e28c4c1f7695b6a9b77af80b3b4973f2" Jul 06 23:53:36.686049 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.686008 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9d45bf4550d91de919e1b2bae371c51e28c4c1f7695b6a9b77af80b3b4973f2"} err="failed to get container status \"f9d45bf4550d91de919e1b2bae371c51e28c4c1f7695b6a9b77af80b3b4973f2\": rpc error: code = NotFound desc = could not find container \"f9d45bf4550d91de919e1b2bae371c51e28c4c1f7695b6a9b77af80b3b4973f2\": container with ID starting with f9d45bf4550d91de919e1b2bae371c51e28c4c1f7695b6a9b77af80b3b4973f2 not found: ID does not exist" Jul 06 23:53:36.693821 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.693795 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-676b998b6d-ct9fd"] Jul 06 23:53:36.697104 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:36.697081 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-676b998b6d-ct9fd"] Jul 06 23:53:38.150805 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:38.150771 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4z8vq_3253bdfe-5248-4d4c-9864-11660eb042ca/ovn-acl-logging/0.log" Jul 06 23:53:38.154357 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:38.154334 2553 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 23:53:38.159401 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:38.159378 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4z8vq_3253bdfe-5248-4d4c-9864-11660eb042ca/ovn-acl-logging/0.log" Jul 06 23:53:38.301894 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:53:38.301748 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a45a3275-3d71-4c1b-9a17-82174da36004" path="/var/lib/kubelet/pods/a45a3275-3d71-4c1b-9a17-82174da36004/volumes" Jul 06 23:54:02.344185 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:02.344149 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts"] Jul 06 23:54:02.344629 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:02.344461 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a45a3275-3d71-4c1b-9a17-82174da36004" containerName="console" Jul 06 23:54:02.344629 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:02.344475 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="a45a3275-3d71-4c1b-9a17-82174da36004" containerName="console" Jul 06 23:54:02.344629 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:02.344529 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="a45a3275-3d71-4c1b-9a17-82174da36004" containerName="console" Jul 06 23:54:02.347669 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:02.347642 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts" Jul 06 23:54:02.350973 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:02.350950 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 23:54:02.351496 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:02.351478 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 23:54:02.352415 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:02.352396 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-72h2q\"" Jul 06 23:54:02.373907 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:02.373869 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts"] Jul 06 23:54:02.374170 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:02.374147 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj2v4\" (UniqueName: \"kubernetes.io/projected/c4536579-e3bd-48ad-b84b-e11987d0e47d-kube-api-access-vj2v4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts\" (UID: \"c4536579-e3bd-48ad-b84b-e11987d0e47d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts" Jul 06 23:54:02.374261 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:02.374239 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4536579-e3bd-48ad-b84b-e11987d0e47d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts\" (UID: \"c4536579-e3bd-48ad-b84b-e11987d0e47d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts" Jul 06 23:54:02.374416 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:02.374393 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4536579-e3bd-48ad-b84b-e11987d0e47d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts\" (UID: \"c4536579-e3bd-48ad-b84b-e11987d0e47d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts" Jul 06 23:54:02.475171 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:02.475129 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4536579-e3bd-48ad-b84b-e11987d0e47d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts\" (UID: \"c4536579-e3bd-48ad-b84b-e11987d0e47d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts" Jul 06 23:54:02.475413 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:02.475179 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj2v4\" (UniqueName: \"kubernetes.io/projected/c4536579-e3bd-48ad-b84b-e11987d0e47d-kube-api-access-vj2v4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts\" (UID: \"c4536579-e3bd-48ad-b84b-e11987d0e47d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts" Jul 06 23:54:02.475413 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:02.475232 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4536579-e3bd-48ad-b84b-e11987d0e47d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts\" (UID: \"c4536579-e3bd-48ad-b84b-e11987d0e47d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts" Jul 06 23:54:02.475630 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:02.475605 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4536579-e3bd-48ad-b84b-e11987d0e47d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts\" (UID: \"c4536579-e3bd-48ad-b84b-e11987d0e47d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts" Jul 06 23:54:02.475694 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:02.475619 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4536579-e3bd-48ad-b84b-e11987d0e47d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts\" (UID: \"c4536579-e3bd-48ad-b84b-e11987d0e47d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts" Jul 06 23:54:02.483537 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:02.483512 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj2v4\" (UniqueName: \"kubernetes.io/projected/c4536579-e3bd-48ad-b84b-e11987d0e47d-kube-api-access-vj2v4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts\" (UID: \"c4536579-e3bd-48ad-b84b-e11987d0e47d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts" Jul 06 23:54:02.657176 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:02.657144 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts" Jul 06 23:54:02.793493 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:02.793455 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts"] Jul 06 23:54:02.796458 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:54:02.796431 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4536579_e3bd_48ad_b84b_e11987d0e47d.slice/crio-afaa5c72a55e5ba37bc1fa532d638044b5c7c19e763bcc853eab091b2975a28c WatchSource:0}: Error finding container afaa5c72a55e5ba37bc1fa532d638044b5c7c19e763bcc853eab091b2975a28c: Status 404 returned error can't find the container with id afaa5c72a55e5ba37bc1fa532d638044b5c7c19e763bcc853eab091b2975a28c Jul 06 23:54:02.798756 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:02.798739 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 23:54:03.775555 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:03.775507 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts" event={"ID":"c4536579-e3bd-48ad-b84b-e11987d0e47d","Type":"ContainerStarted","Data":"afaa5c72a55e5ba37bc1fa532d638044b5c7c19e763bcc853eab091b2975a28c"} Jul 06 23:54:08.797266 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:08.797224 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts" event={"ID":"c4536579-e3bd-48ad-b84b-e11987d0e47d","Type":"ContainerStarted","Data":"ef8b33a54f4861916e8a76e3977ef5aebabd99f42025103ffb48763f9e2507c3"} Jul 06 23:54:09.802576 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:09.802537 2553 generic.go:356] "Generic (PLEG): container finished" podID="c4536579-e3bd-48ad-b84b-e11987d0e47d" containerID="ef8b33a54f4861916e8a76e3977ef5aebabd99f42025103ffb48763f9e2507c3" exitCode=0 Jul 06 23:54:09.803080 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:09.802607 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts" event={"ID":"c4536579-e3bd-48ad-b84b-e11987d0e47d","Type":"ContainerDied","Data":"ef8b33a54f4861916e8a76e3977ef5aebabd99f42025103ffb48763f9e2507c3"} Jul 06 23:54:12.816183 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:12.816145 2553 generic.go:356] "Generic (PLEG): container finished" podID="c4536579-e3bd-48ad-b84b-e11987d0e47d" containerID="1ba37a9203a89976564d6f81464501e07cb35584ee13a40b3657dbe02d7c3982" exitCode=0 Jul 06 23:54:12.816621 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:12.816230 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts" event={"ID":"c4536579-e3bd-48ad-b84b-e11987d0e47d","Type":"ContainerDied","Data":"1ba37a9203a89976564d6f81464501e07cb35584ee13a40b3657dbe02d7c3982"} Jul 06 23:54:13.390763 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:54:13.390719 2553 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:97169107e88251ef86278e9e44644664b531b5472896c4647d9828f2a9e32e40" Jul 06 23:54:13.390940 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:54:13.390833 2553 kuberuntime_manager.go:1459] "Unhandled Error" err="container extract start failed in pod 685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts_openshift-marketplace(c4536579-e3bd-48ad-b84b-e11987d0e47d): ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" logger="UnhandledError" Jul 06 23:54:13.390940 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:54:13.390867 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"extract\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts" podUID="c4536579-e3bd-48ad-b84b-e11987d0e47d" Jul 06 23:54:13.821195 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:54:13.821167 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"extract\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:97169107e88251ef86278e9e44644664b531b5472896c4647d9828f2a9e32e40\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts" podUID="c4536579-e3bd-48ad-b84b-e11987d0e47d" Jul 06 23:54:34.904786 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:34.904745 2553 generic.go:356] "Generic (PLEG): container finished" podID="c4536579-e3bd-48ad-b84b-e11987d0e47d" containerID="301a8b7f2937a3e1cc98ed5fc5da946758d350e9037e1bcd45bb258c71ae62e5" exitCode=0 Jul 06 23:54:34.905178 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:34.904813 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts" event={"ID":"c4536579-e3bd-48ad-b84b-e11987d0e47d","Type":"ContainerDied","Data":"301a8b7f2937a3e1cc98ed5fc5da946758d350e9037e1bcd45bb258c71ae62e5"} Jul 06 23:54:36.031908 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:36.031877 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts" Jul 06 23:54:36.165653 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:36.165556 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vj2v4\" (UniqueName: \"kubernetes.io/projected/c4536579-e3bd-48ad-b84b-e11987d0e47d-kube-api-access-vj2v4\") pod \"c4536579-e3bd-48ad-b84b-e11987d0e47d\" (UID: \"c4536579-e3bd-48ad-b84b-e11987d0e47d\") " Jul 06 23:54:36.165653 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:36.165599 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4536579-e3bd-48ad-b84b-e11987d0e47d-bundle\") pod \"c4536579-e3bd-48ad-b84b-e11987d0e47d\" (UID: \"c4536579-e3bd-48ad-b84b-e11987d0e47d\") " Jul 06 23:54:36.165653 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:36.165646 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4536579-e3bd-48ad-b84b-e11987d0e47d-util\") pod \"c4536579-e3bd-48ad-b84b-e11987d0e47d\" (UID: \"c4536579-e3bd-48ad-b84b-e11987d0e47d\") " Jul 06 23:54:36.166285 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:36.166252 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4536579-e3bd-48ad-b84b-e11987d0e47d-bundle" (OuterVolumeSpecName: "bundle") pod "c4536579-e3bd-48ad-b84b-e11987d0e47d" (UID: "c4536579-e3bd-48ad-b84b-e11987d0e47d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:54:36.167826 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:36.167793 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4536579-e3bd-48ad-b84b-e11987d0e47d-kube-api-access-vj2v4" (OuterVolumeSpecName: "kube-api-access-vj2v4") pod "c4536579-e3bd-48ad-b84b-e11987d0e47d" (UID: "c4536579-e3bd-48ad-b84b-e11987d0e47d"). InnerVolumeSpecName "kube-api-access-vj2v4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:54:36.169853 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:36.169834 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4536579-e3bd-48ad-b84b-e11987d0e47d-util" (OuterVolumeSpecName: "util") pod "c4536579-e3bd-48ad-b84b-e11987d0e47d" (UID: "c4536579-e3bd-48ad-b84b-e11987d0e47d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:54:36.266979 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:36.266941 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vj2v4\" (UniqueName: \"kubernetes.io/projected/c4536579-e3bd-48ad-b84b-e11987d0e47d-kube-api-access-vj2v4\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:54:36.266979 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:36.266972 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4536579-e3bd-48ad-b84b-e11987d0e47d-bundle\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:54:36.266979 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:36.266981 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4536579-e3bd-48ad-b84b-e11987d0e47d-util\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:54:36.913913 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:36.913877 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts" event={"ID":"c4536579-e3bd-48ad-b84b-e11987d0e47d","Type":"ContainerDied","Data":"afaa5c72a55e5ba37bc1fa532d638044b5c7c19e763bcc853eab091b2975a28c"} Jul 06 23:54:36.914070 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:36.913925 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="afaa5c72a55e5ba37bc1fa532d638044b5c7c19e763bcc853eab091b2975a28c" Jul 06 23:54:36.914070 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:36.913893 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h4jts" Jul 06 23:54:40.762937 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:40.762899 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8ls5z"] Jul 06 23:54:40.763454 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:40.763166 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4536579-e3bd-48ad-b84b-e11987d0e47d" containerName="extract" Jul 06 23:54:40.763454 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:40.763177 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4536579-e3bd-48ad-b84b-e11987d0e47d" containerName="extract" Jul 06 23:54:40.763454 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:40.763187 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4536579-e3bd-48ad-b84b-e11987d0e47d" containerName="pull" Jul 06 23:54:40.763454 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:40.763193 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4536579-e3bd-48ad-b84b-e11987d0e47d" containerName="pull" Jul 06 23:54:40.763454 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:40.763213 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4536579-e3bd-48ad-b84b-e11987d0e47d" containerName="util" Jul 06 23:54:40.763454 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:40.763221 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4536579-e3bd-48ad-b84b-e11987d0e47d" containerName="util" Jul 06 23:54:40.763454 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:40.763266 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4536579-e3bd-48ad-b84b-e11987d0e47d" containerName="extract" Jul 06 23:54:40.765992 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:40.765976 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8ls5z" Jul 06 23:54:40.768795 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:40.768764 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 06 23:54:40.768963 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:40.768889 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-gchg7\"" Jul 06 23:54:40.769233 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:40.769220 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 06 23:54:40.783628 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:40.783599 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8ls5z"] Jul 06 23:54:40.904708 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:40.904671 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe5e9ea9-f3f5-40b1-babf-fd7bfecc3f66-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8ls5z\" (UID: \"fe5e9ea9-f3f5-40b1-babf-fd7bfecc3f66\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8ls5z" Jul 06 23:54:40.904889 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:40.904749 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9sz5\" (UniqueName: \"kubernetes.io/projected/fe5e9ea9-f3f5-40b1-babf-fd7bfecc3f66-kube-api-access-v9sz5\") pod \"cert-manager-operator-controller-manager-55c444bf86-8ls5z\" (UID: \"fe5e9ea9-f3f5-40b1-babf-fd7bfecc3f66\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8ls5z" Jul 06 23:54:41.005447 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:41.005413 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe5e9ea9-f3f5-40b1-babf-fd7bfecc3f66-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8ls5z\" (UID: \"fe5e9ea9-f3f5-40b1-babf-fd7bfecc3f66\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8ls5z" Jul 06 23:54:41.005626 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:41.005485 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9sz5\" (UniqueName: \"kubernetes.io/projected/fe5e9ea9-f3f5-40b1-babf-fd7bfecc3f66-kube-api-access-v9sz5\") pod \"cert-manager-operator-controller-manager-55c444bf86-8ls5z\" (UID: \"fe5e9ea9-f3f5-40b1-babf-fd7bfecc3f66\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8ls5z" Jul 06 23:54:41.005800 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:41.005779 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe5e9ea9-f3f5-40b1-babf-fd7bfecc3f66-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8ls5z\" (UID: \"fe5e9ea9-f3f5-40b1-babf-fd7bfecc3f66\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8ls5z" Jul 06 23:54:41.014258 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:41.014192 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9sz5\" (UniqueName: \"kubernetes.io/projected/fe5e9ea9-f3f5-40b1-babf-fd7bfecc3f66-kube-api-access-v9sz5\") pod \"cert-manager-operator-controller-manager-55c444bf86-8ls5z\" (UID: \"fe5e9ea9-f3f5-40b1-babf-fd7bfecc3f66\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8ls5z" Jul 06 23:54:41.075184 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:41.075146 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8ls5z" Jul 06 23:54:41.211915 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:41.211765 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8ls5z"] Jul 06 23:54:41.214806 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:54:41.214780 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe5e9ea9_f3f5_40b1_babf_fd7bfecc3f66.slice/crio-d504b9ae5a2b726440d60a276aac29f9734bee9ce03f3ef5dad7d19825e3b8ee WatchSource:0}: Error finding container d504b9ae5a2b726440d60a276aac29f9734bee9ce03f3ef5dad7d19825e3b8ee: Status 404 returned error can't find the container with id d504b9ae5a2b726440d60a276aac29f9734bee9ce03f3ef5dad7d19825e3b8ee Jul 06 23:54:41.936212 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:41.936175 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8ls5z" event={"ID":"fe5e9ea9-f3f5-40b1-babf-fd7bfecc3f66","Type":"ContainerStarted","Data":"d504b9ae5a2b726440d60a276aac29f9734bee9ce03f3ef5dad7d19825e3b8ee"} Jul 06 23:54:43.946146 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:43.946108 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8ls5z" event={"ID":"fe5e9ea9-f3f5-40b1-babf-fd7bfecc3f66","Type":"ContainerStarted","Data":"d607698a39f3c7df663c8ffdf1c3a713204239082ad5cd059e383735337bdd16"} Jul 06 23:54:43.968343 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:43.968278 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8ls5z" podStartSLOduration=2.2235247 podStartE2EDuration="3.968261998s" totalImagesPullingTime="1.744737298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:54:43.966199677 +0000 UTC m=+366.175929197" watchObservedRunningTime="2026-07-06 23:54:43.968261998 +0000 UTC m=+366.177991518" Jul 06 23:54:45.184234 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:45.184196 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx"] Jul 06 23:54:45.187668 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:45.187652 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx" Jul 06 23:54:45.190393 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:45.190361 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 23:54:45.190393 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:45.190367 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-72h2q\"" Jul 06 23:54:45.190719 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:45.190700 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 23:54:45.215001 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:45.214964 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx"] Jul 06 23:54:45.341669 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:45.341637 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/19d1fe39-f85e-4d16-ae43-6c0a854870ec-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx\" (UID: \"19d1fe39-f85e-4d16-ae43-6c0a854870ec\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx" Jul 06 23:54:45.341669 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:45.341675 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lvgz\" (UniqueName: \"kubernetes.io/projected/19d1fe39-f85e-4d16-ae43-6c0a854870ec-kube-api-access-5lvgz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx\" (UID: \"19d1fe39-f85e-4d16-ae43-6c0a854870ec\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx" Jul 06 23:54:45.341903 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:45.341699 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/19d1fe39-f85e-4d16-ae43-6c0a854870ec-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx\" (UID: \"19d1fe39-f85e-4d16-ae43-6c0a854870ec\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx" Jul 06 23:54:45.442608 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:45.442519 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/19d1fe39-f85e-4d16-ae43-6c0a854870ec-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx\" (UID: \"19d1fe39-f85e-4d16-ae43-6c0a854870ec\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx" Jul 06 23:54:45.442608 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:45.442569 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lvgz\" (UniqueName: \"kubernetes.io/projected/19d1fe39-f85e-4d16-ae43-6c0a854870ec-kube-api-access-5lvgz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx\" (UID: \"19d1fe39-f85e-4d16-ae43-6c0a854870ec\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx" Jul 06 23:54:45.442608 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:45.442597 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/19d1fe39-f85e-4d16-ae43-6c0a854870ec-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx\" (UID: \"19d1fe39-f85e-4d16-ae43-6c0a854870ec\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx" Jul 06 23:54:45.442877 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:45.442857 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/19d1fe39-f85e-4d16-ae43-6c0a854870ec-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx\" (UID: \"19d1fe39-f85e-4d16-ae43-6c0a854870ec\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx" Jul 06 23:54:45.442935 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:45.442915 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/19d1fe39-f85e-4d16-ae43-6c0a854870ec-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx\" (UID: \"19d1fe39-f85e-4d16-ae43-6c0a854870ec\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx" Jul 06 23:54:45.452386 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:45.452359 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lvgz\" (UniqueName: \"kubernetes.io/projected/19d1fe39-f85e-4d16-ae43-6c0a854870ec-kube-api-access-5lvgz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx\" (UID: \"19d1fe39-f85e-4d16-ae43-6c0a854870ec\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx" Jul 06 23:54:45.497234 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:45.497190 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx" Jul 06 23:54:45.641691 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:45.641651 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx"] Jul 06 23:54:45.645012 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:54:45.644982 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19d1fe39_f85e_4d16_ae43_6c0a854870ec.slice/crio-b4ae23a60c9ff2d52ac0bc0ebc9244cbdebbb0b0a82662a7a6f92a5ae72a0d09 WatchSource:0}: Error finding container b4ae23a60c9ff2d52ac0bc0ebc9244cbdebbb0b0a82662a7a6f92a5ae72a0d09: Status 404 returned error can't find the container with id b4ae23a60c9ff2d52ac0bc0ebc9244cbdebbb0b0a82662a7a6f92a5ae72a0d09 Jul 06 23:54:45.955205 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:45.955111 2553 generic.go:356] "Generic (PLEG): container finished" podID="19d1fe39-f85e-4d16-ae43-6c0a854870ec" containerID="ab5fcc030b50a2b52b90a9d2ebd8791b4bf61644f0ed62940ef10dd7c349e9a5" exitCode=0 Jul 06 23:54:45.955205 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:45.955150 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx" event={"ID":"19d1fe39-f85e-4d16-ae43-6c0a854870ec","Type":"ContainerDied","Data":"ab5fcc030b50a2b52b90a9d2ebd8791b4bf61644f0ed62940ef10dd7c349e9a5"} Jul 06 23:54:45.955205 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:45.955186 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx" event={"ID":"19d1fe39-f85e-4d16-ae43-6c0a854870ec","Type":"ContainerStarted","Data":"b4ae23a60c9ff2d52ac0bc0ebc9244cbdebbb0b0a82662a7a6f92a5ae72a0d09"} Jul 06 23:54:48.045261 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:48.045228 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-qmsvs"] Jul 06 23:54:48.048507 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:48.048484 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-qmsvs" Jul 06 23:54:48.051383 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:48.051353 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 23:54:48.052559 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:48.052540 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-fkcfx\"" Jul 06 23:54:48.052672 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:48.052559 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 23:54:48.062231 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:48.062205 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-qmsvs"] Jul 06 23:54:48.164558 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:48.164514 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dr7t\" (UniqueName: \"kubernetes.io/projected/f8193619-0e53-4c16-b292-d8309d444331-kube-api-access-4dr7t\") pod \"cert-manager-cainjector-559cdd8c87-qmsvs\" (UID: \"f8193619-0e53-4c16-b292-d8309d444331\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qmsvs" Jul 06 23:54:48.164739 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:48.164618 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8193619-0e53-4c16-b292-d8309d444331-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-qmsvs\" (UID: \"f8193619-0e53-4c16-b292-d8309d444331\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qmsvs" Jul 06 23:54:48.265665 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:48.265569 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dr7t\" (UniqueName: \"kubernetes.io/projected/f8193619-0e53-4c16-b292-d8309d444331-kube-api-access-4dr7t\") pod \"cert-manager-cainjector-559cdd8c87-qmsvs\" (UID: \"f8193619-0e53-4c16-b292-d8309d444331\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qmsvs" Jul 06 23:54:48.265665 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:48.265631 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8193619-0e53-4c16-b292-d8309d444331-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-qmsvs\" (UID: \"f8193619-0e53-4c16-b292-d8309d444331\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qmsvs" Jul 06 23:54:48.275623 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:48.275589 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8193619-0e53-4c16-b292-d8309d444331-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-qmsvs\" (UID: \"f8193619-0e53-4c16-b292-d8309d444331\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qmsvs" Jul 06 23:54:48.275747 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:48.275640 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dr7t\" (UniqueName: \"kubernetes.io/projected/f8193619-0e53-4c16-b292-d8309d444331-kube-api-access-4dr7t\") pod \"cert-manager-cainjector-559cdd8c87-qmsvs\" (UID: \"f8193619-0e53-4c16-b292-d8309d444331\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qmsvs" Jul 06 23:54:48.367358 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:48.367331 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-qmsvs" Jul 06 23:54:48.518687 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:48.518661 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-qmsvs"] Jul 06 23:54:48.520210 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:54:48.520181 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf8193619_0e53_4c16_b292_d8309d444331.slice/crio-86a3f38f3f7a768f824f00e29730a8544a2efa77df4c8663b20d9b2cc391c3e6 WatchSource:0}: Error finding container 86a3f38f3f7a768f824f00e29730a8544a2efa77df4c8663b20d9b2cc391c3e6: Status 404 returned error can't find the container with id 86a3f38f3f7a768f824f00e29730a8544a2efa77df4c8663b20d9b2cc391c3e6 Jul 06 23:54:48.975187 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:48.975149 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-qmsvs" event={"ID":"f8193619-0e53-4c16-b292-d8309d444331","Type":"ContainerStarted","Data":"86a3f38f3f7a768f824f00e29730a8544a2efa77df4c8663b20d9b2cc391c3e6"} Jul 06 23:54:48.977016 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:48.976990 2553 generic.go:356] "Generic (PLEG): container finished" podID="19d1fe39-f85e-4d16-ae43-6c0a854870ec" containerID="4578a31bf1f6e5eea0f5adf1b967e08434ace13a27b7f7881ceeb729e1751de3" exitCode=0 Jul 06 23:54:48.977124 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:48.977034 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx" event={"ID":"19d1fe39-f85e-4d16-ae43-6c0a854870ec","Type":"ContainerDied","Data":"4578a31bf1f6e5eea0f5adf1b967e08434ace13a27b7f7881ceeb729e1751de3"} Jul 06 23:54:49.071480 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:54:49.071442 2553 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19d1fe39_f85e_4d16_ae43_6c0a854870ec.slice/crio-f40e83bbca1f4b75c2f0d5e77a206d6d17a3702a280931cd291f350dd7063711.scope\": RecentStats: unable to find data in memory cache]" Jul 06 23:54:49.985100 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:49.985056 2553 generic.go:356] "Generic (PLEG): container finished" podID="19d1fe39-f85e-4d16-ae43-6c0a854870ec" containerID="f40e83bbca1f4b75c2f0d5e77a206d6d17a3702a280931cd291f350dd7063711" exitCode=0 Jul 06 23:54:49.985285 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:49.985129 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx" event={"ID":"19d1fe39-f85e-4d16-ae43-6c0a854870ec","Type":"ContainerDied","Data":"f40e83bbca1f4b75c2f0d5e77a206d6d17a3702a280931cd291f350dd7063711"} Jul 06 23:54:51.132481 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:51.132457 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx" Jul 06 23:54:51.289518 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:51.289431 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/19d1fe39-f85e-4d16-ae43-6c0a854870ec-bundle\") pod \"19d1fe39-f85e-4d16-ae43-6c0a854870ec\" (UID: \"19d1fe39-f85e-4d16-ae43-6c0a854870ec\") " Jul 06 23:54:51.289518 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:51.289470 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lvgz\" (UniqueName: \"kubernetes.io/projected/19d1fe39-f85e-4d16-ae43-6c0a854870ec-kube-api-access-5lvgz\") pod \"19d1fe39-f85e-4d16-ae43-6c0a854870ec\" (UID: \"19d1fe39-f85e-4d16-ae43-6c0a854870ec\") " Jul 06 23:54:51.289518 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:51.289518 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/19d1fe39-f85e-4d16-ae43-6c0a854870ec-util\") pod \"19d1fe39-f85e-4d16-ae43-6c0a854870ec\" (UID: \"19d1fe39-f85e-4d16-ae43-6c0a854870ec\") " Jul 06 23:54:51.289892 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:51.289853 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19d1fe39-f85e-4d16-ae43-6c0a854870ec-bundle" (OuterVolumeSpecName: "bundle") pod "19d1fe39-f85e-4d16-ae43-6c0a854870ec" (UID: "19d1fe39-f85e-4d16-ae43-6c0a854870ec"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:54:51.291740 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:51.291714 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19d1fe39-f85e-4d16-ae43-6c0a854870ec-kube-api-access-5lvgz" (OuterVolumeSpecName: "kube-api-access-5lvgz") pod "19d1fe39-f85e-4d16-ae43-6c0a854870ec" (UID: "19d1fe39-f85e-4d16-ae43-6c0a854870ec"). InnerVolumeSpecName "kube-api-access-5lvgz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:54:51.295550 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:51.295518 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19d1fe39-f85e-4d16-ae43-6c0a854870ec-util" (OuterVolumeSpecName: "util") pod "19d1fe39-f85e-4d16-ae43-6c0a854870ec" (UID: "19d1fe39-f85e-4d16-ae43-6c0a854870ec"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:54:51.390120 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:51.390074 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/19d1fe39-f85e-4d16-ae43-6c0a854870ec-util\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:54:51.390416 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:51.390388 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/19d1fe39-f85e-4d16-ae43-6c0a854870ec-bundle\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:54:51.390416 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:51.390415 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lvgz\" (UniqueName: \"kubernetes.io/projected/19d1fe39-f85e-4d16-ae43-6c0a854870ec-kube-api-access-5lvgz\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:54:51.994787 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:51.994753 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-qmsvs" event={"ID":"f8193619-0e53-4c16-b292-d8309d444331","Type":"ContainerStarted","Data":"87a61bbbe7e666eb3b7b6a45e49bca5a744a1663c5779731b60c6649d97ef3d7"} Jul 06 23:54:51.996633 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:51.996605 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx" event={"ID":"19d1fe39-f85e-4d16-ae43-6c0a854870ec","Type":"ContainerDied","Data":"b4ae23a60c9ff2d52ac0bc0ebc9244cbdebbb0b0a82662a7a6f92a5ae72a0d09"} Jul 06 23:54:51.996779 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:51.996642 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b4ae23a60c9ff2d52ac0bc0ebc9244cbdebbb0b0a82662a7a6f92a5ae72a0d09" Jul 06 23:54:51.996779 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:51.996655 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkb6qx" Jul 06 23:54:52.012769 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:52.012716 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-qmsvs" podStartSLOduration=1.41105997 podStartE2EDuration="4.012700945s" totalImagesPullingTime="2.601640975s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:54:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:54:52.011917081 +0000 UTC m=+374.221646616" watchObservedRunningTime="2026-07-06 23:54:52.012700945 +0000 UTC m=+374.222430465" Jul 06 23:54:52.184072 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:52.184036 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7n6ww"] Jul 06 23:54:52.184578 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:52.184509 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19d1fe39-f85e-4d16-ae43-6c0a854870ec" containerName="pull" Jul 06 23:54:52.184578 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:52.184529 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="19d1fe39-f85e-4d16-ae43-6c0a854870ec" containerName="pull" Jul 06 23:54:52.184578 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:52.184548 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19d1fe39-f85e-4d16-ae43-6c0a854870ec" containerName="util" Jul 06 23:54:52.184578 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:52.184556 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="19d1fe39-f85e-4d16-ae43-6c0a854870ec" containerName="util" Jul 06 23:54:52.184578 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:52.184578 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19d1fe39-f85e-4d16-ae43-6c0a854870ec" containerName="extract" Jul 06 23:54:52.184841 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:52.184587 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="19d1fe39-f85e-4d16-ae43-6c0a854870ec" containerName="extract" Jul 06 23:54:52.184841 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:52.184656 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="19d1fe39-f85e-4d16-ae43-6c0a854870ec" containerName="extract" Jul 06 23:54:52.188026 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:52.188005 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-7n6ww" Jul 06 23:54:52.190449 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:52.190414 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-6475q\"" Jul 06 23:54:52.196165 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:52.196142 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7n6ww"] Jul 06 23:54:52.296880 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:52.296786 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2a594e64-9812-4229-b802-0a049c7729ec-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7n6ww\" (UID: \"2a594e64-9812-4229-b802-0a049c7729ec\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7n6ww" Jul 06 23:54:52.296880 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:52.296855 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5fkl\" (UniqueName: \"kubernetes.io/projected/2a594e64-9812-4229-b802-0a049c7729ec-kube-api-access-x5fkl\") pod \"cert-manager-webhook-6486ffd997-7n6ww\" (UID: \"2a594e64-9812-4229-b802-0a049c7729ec\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7n6ww" Jul 06 23:54:52.397834 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:52.397480 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2a594e64-9812-4229-b802-0a049c7729ec-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7n6ww\" (UID: \"2a594e64-9812-4229-b802-0a049c7729ec\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7n6ww" Jul 06 23:54:52.397834 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:52.397556 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5fkl\" (UniqueName: \"kubernetes.io/projected/2a594e64-9812-4229-b802-0a049c7729ec-kube-api-access-x5fkl\") pod \"cert-manager-webhook-6486ffd997-7n6ww\" (UID: \"2a594e64-9812-4229-b802-0a049c7729ec\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7n6ww" Jul 06 23:54:52.406998 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:52.406966 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2a594e64-9812-4229-b802-0a049c7729ec-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7n6ww\" (UID: \"2a594e64-9812-4229-b802-0a049c7729ec\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7n6ww" Jul 06 23:54:52.407120 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:52.407016 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5fkl\" (UniqueName: \"kubernetes.io/projected/2a594e64-9812-4229-b802-0a049c7729ec-kube-api-access-x5fkl\") pod \"cert-manager-webhook-6486ffd997-7n6ww\" (UID: \"2a594e64-9812-4229-b802-0a049c7729ec\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7n6ww" Jul 06 23:54:52.497938 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:52.497898 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-7n6ww" Jul 06 23:54:52.628685 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:52.628596 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7n6ww"] Jul 06 23:54:52.631528 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:54:52.631492 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2a594e64_9812_4229_b802_0a049c7729ec.slice/crio-60b4d6bca27574abec4ad5e7a4f364af5cb46833174e4dfb5f2e9c8083ac28fa WatchSource:0}: Error finding container 60b4d6bca27574abec4ad5e7a4f364af5cb46833174e4dfb5f2e9c8083ac28fa: Status 404 returned error can't find the container with id 60b4d6bca27574abec4ad5e7a4f364af5cb46833174e4dfb5f2e9c8083ac28fa Jul 06 23:54:53.001913 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:53.001871 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-7n6ww" event={"ID":"2a594e64-9812-4229-b802-0a049c7729ec","Type":"ContainerStarted","Data":"4fccb3a721313c9e85e8822374154da58eeb78e44f4abbfbac21d6ae2552f510"} Jul 06 23:54:53.002100 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:53.001923 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-7n6ww" event={"ID":"2a594e64-9812-4229-b802-0a049c7729ec","Type":"ContainerStarted","Data":"60b4d6bca27574abec4ad5e7a4f364af5cb46833174e4dfb5f2e9c8083ac28fa"} Jul 06 23:54:53.002100 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:53.001965 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-7n6ww" Jul 06 23:54:53.022374 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:53.022287 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-7n6ww" podStartSLOduration=1.022155235 podStartE2EDuration="1.022155235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:54:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:54:53.02121234 +0000 UTC m=+375.230941864" watchObservedRunningTime="2026-07-06 23:54:53.022155235 +0000 UTC m=+375.231884757" Jul 06 23:54:57.101230 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:57.101168 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-mgr4c"] Jul 06 23:54:57.106585 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:57.106553 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-mgr4c" Jul 06 23:54:57.109980 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:57.109957 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-54l5z\"" Jul 06 23:54:57.125724 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:57.125689 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-mgr4c"] Jul 06 23:54:57.132315 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:57.132270 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f57e6233-2ec4-4b5e-b404-3da9edffc7a7-bound-sa-token\") pod \"cert-manager-55b4d954d5-mgr4c\" (UID: \"f57e6233-2ec4-4b5e-b404-3da9edffc7a7\") " pod="cert-manager/cert-manager-55b4d954d5-mgr4c" Jul 06 23:54:57.132481 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:57.132326 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lwnb\" (UniqueName: \"kubernetes.io/projected/f57e6233-2ec4-4b5e-b404-3da9edffc7a7-kube-api-access-7lwnb\") pod \"cert-manager-55b4d954d5-mgr4c\" (UID: \"f57e6233-2ec4-4b5e-b404-3da9edffc7a7\") " pod="cert-manager/cert-manager-55b4d954d5-mgr4c" Jul 06 23:54:57.233364 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:57.233328 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f57e6233-2ec4-4b5e-b404-3da9edffc7a7-bound-sa-token\") pod \"cert-manager-55b4d954d5-mgr4c\" (UID: \"f57e6233-2ec4-4b5e-b404-3da9edffc7a7\") " pod="cert-manager/cert-manager-55b4d954d5-mgr4c" Jul 06 23:54:57.233364 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:57.233369 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lwnb\" (UniqueName: \"kubernetes.io/projected/f57e6233-2ec4-4b5e-b404-3da9edffc7a7-kube-api-access-7lwnb\") pod \"cert-manager-55b4d954d5-mgr4c\" (UID: \"f57e6233-2ec4-4b5e-b404-3da9edffc7a7\") " pod="cert-manager/cert-manager-55b4d954d5-mgr4c" Jul 06 23:54:57.242595 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:57.242560 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lwnb\" (UniqueName: \"kubernetes.io/projected/f57e6233-2ec4-4b5e-b404-3da9edffc7a7-kube-api-access-7lwnb\") pod \"cert-manager-55b4d954d5-mgr4c\" (UID: \"f57e6233-2ec4-4b5e-b404-3da9edffc7a7\") " pod="cert-manager/cert-manager-55b4d954d5-mgr4c" Jul 06 23:54:57.243992 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:57.243963 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f57e6233-2ec4-4b5e-b404-3da9edffc7a7-bound-sa-token\") pod \"cert-manager-55b4d954d5-mgr4c\" (UID: \"f57e6233-2ec4-4b5e-b404-3da9edffc7a7\") " pod="cert-manager/cert-manager-55b4d954d5-mgr4c" Jul 06 23:54:57.416966 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:57.416932 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-mgr4c" Jul 06 23:54:57.554028 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:57.553991 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-mgr4c"] Jul 06 23:54:57.556922 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:54:57.556888 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf57e6233_2ec4_4b5e_b404_3da9edffc7a7.slice/crio-ad5a1a331e33b1f2764b23c4eef8d5b013416dbbf0ae469c099eee00eb831b0e WatchSource:0}: Error finding container ad5a1a331e33b1f2764b23c4eef8d5b013416dbbf0ae469c099eee00eb831b0e: Status 404 returned error can't find the container with id ad5a1a331e33b1f2764b23c4eef8d5b013416dbbf0ae469c099eee00eb831b0e Jul 06 23:54:58.024137 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:58.024094 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-mgr4c" event={"ID":"f57e6233-2ec4-4b5e-b404-3da9edffc7a7","Type":"ContainerStarted","Data":"0eef1830d2720a4e25c0ea9e1cf8a44b680c40e12862307dd36e60e01cf9fab9"} Jul 06 23:54:58.024325 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:58.024151 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-mgr4c" event={"ID":"f57e6233-2ec4-4b5e-b404-3da9edffc7a7","Type":"ContainerStarted","Data":"ad5a1a331e33b1f2764b23c4eef8d5b013416dbbf0ae469c099eee00eb831b0e"} Jul 06 23:54:58.045438 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:58.045385 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-mgr4c" podStartSLOduration=1.045370122 podStartE2EDuration="1.045370122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:54:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:54:58.044542618 +0000 UTC m=+380.254272140" watchObservedRunningTime="2026-07-06 23:54:58.045370122 +0000 UTC m=+380.255099643" Jul 06 23:54:59.008236 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:54:59.008205 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-7n6ww" Jul 06 23:55:04.420172 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:04.420129 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7"] Jul 06 23:55:04.439396 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:04.439358 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7"] Jul 06 23:55:04.439550 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:04.439439 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7" Jul 06 23:55:04.442574 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:04.442548 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 23:55:04.443736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:04.443707 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 23:55:04.443872 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:04.443758 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-72h2q\"" Jul 06 23:55:04.490813 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:04.490770 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd24f9bd-8033-41ce-926f-543d2b190bad-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7\" (UID: \"cd24f9bd-8033-41ce-926f-543d2b190bad\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7" Jul 06 23:55:04.490966 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:04.490846 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqcvp\" (UniqueName: \"kubernetes.io/projected/cd24f9bd-8033-41ce-926f-543d2b190bad-kube-api-access-zqcvp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7\" (UID: \"cd24f9bd-8033-41ce-926f-543d2b190bad\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7" Jul 06 23:55:04.490966 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:04.490906 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd24f9bd-8033-41ce-926f-543d2b190bad-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7\" (UID: \"cd24f9bd-8033-41ce-926f-543d2b190bad\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7" Jul 06 23:55:04.592233 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:04.592130 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd24f9bd-8033-41ce-926f-543d2b190bad-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7\" (UID: \"cd24f9bd-8033-41ce-926f-543d2b190bad\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7" Jul 06 23:55:04.592463 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:04.592325 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqcvp\" (UniqueName: \"kubernetes.io/projected/cd24f9bd-8033-41ce-926f-543d2b190bad-kube-api-access-zqcvp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7\" (UID: \"cd24f9bd-8033-41ce-926f-543d2b190bad\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7" Jul 06 23:55:04.592463 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:04.592365 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd24f9bd-8033-41ce-926f-543d2b190bad-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7\" (UID: \"cd24f9bd-8033-41ce-926f-543d2b190bad\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7" Jul 06 23:55:04.592591 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:04.592568 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd24f9bd-8033-41ce-926f-543d2b190bad-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7\" (UID: \"cd24f9bd-8033-41ce-926f-543d2b190bad\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7" Jul 06 23:55:04.592662 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:04.592645 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd24f9bd-8033-41ce-926f-543d2b190bad-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7\" (UID: \"cd24f9bd-8033-41ce-926f-543d2b190bad\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7" Jul 06 23:55:04.601116 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:04.601082 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqcvp\" (UniqueName: \"kubernetes.io/projected/cd24f9bd-8033-41ce-926f-543d2b190bad-kube-api-access-zqcvp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7\" (UID: \"cd24f9bd-8033-41ce-926f-543d2b190bad\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7" Jul 06 23:55:04.750000 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:04.749903 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7" Jul 06 23:55:04.901074 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:04.901039 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7"] Jul 06 23:55:04.904211 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:55:04.904178 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd24f9bd_8033_41ce_926f_543d2b190bad.slice/crio-d6a838e23485a5eaac4fdd1f5a8d8e96681ea813d73330d9c9a5cab76c54552b WatchSource:0}: Error finding container d6a838e23485a5eaac4fdd1f5a8d8e96681ea813d73330d9c9a5cab76c54552b: Status 404 returned error can't find the container with id d6a838e23485a5eaac4fdd1f5a8d8e96681ea813d73330d9c9a5cab76c54552b Jul 06 23:55:05.060408 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:05.060363 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7" event={"ID":"cd24f9bd-8033-41ce-926f-543d2b190bad","Type":"ContainerStarted","Data":"61e241b3f54efc3d2cd68cf25a87f0d580820154080e741f7a8c34c1d42db589"} Jul 06 23:55:05.060609 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:05.060436 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7" event={"ID":"cd24f9bd-8033-41ce-926f-543d2b190bad","Type":"ContainerStarted","Data":"d6a838e23485a5eaac4fdd1f5a8d8e96681ea813d73330d9c9a5cab76c54552b"} Jul 06 23:55:06.065894 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:06.065858 2553 generic.go:356] "Generic (PLEG): container finished" podID="cd24f9bd-8033-41ce-926f-543d2b190bad" containerID="61e241b3f54efc3d2cd68cf25a87f0d580820154080e741f7a8c34c1d42db589" exitCode=0 Jul 06 23:55:06.066331 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:06.065939 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7" event={"ID":"cd24f9bd-8033-41ce-926f-543d2b190bad","Type":"ContainerDied","Data":"61e241b3f54efc3d2cd68cf25a87f0d580820154080e741f7a8c34c1d42db589"} Jul 06 23:55:07.073121 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:07.073084 2553 generic.go:356] "Generic (PLEG): container finished" podID="cd24f9bd-8033-41ce-926f-543d2b190bad" containerID="ba59c0045120bb8d6ba71d81699e9c1cb528835a11bd63929f5ee3be9ecc15e7" exitCode=0 Jul 06 23:55:07.073598 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:07.073127 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7" event={"ID":"cd24f9bd-8033-41ce-926f-543d2b190bad","Type":"ContainerDied","Data":"ba59c0045120bb8d6ba71d81699e9c1cb528835a11bd63929f5ee3be9ecc15e7"} Jul 06 23:55:08.081712 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:08.081679 2553 generic.go:356] "Generic (PLEG): container finished" podID="cd24f9bd-8033-41ce-926f-543d2b190bad" containerID="f9473da0b0d7adf1a8e67100f0b9a7a1b4324acad6fed4fa27c213a4fa2040cf" exitCode=0 Jul 06 23:55:08.082083 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:08.081762 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7" event={"ID":"cd24f9bd-8033-41ce-926f-543d2b190bad","Type":"ContainerDied","Data":"f9473da0b0d7adf1a8e67100f0b9a7a1b4324acad6fed4fa27c213a4fa2040cf"} Jul 06 23:55:09.213479 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:09.213453 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7" Jul 06 23:55:09.331407 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:09.331368 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd24f9bd-8033-41ce-926f-543d2b190bad-util\") pod \"cd24f9bd-8033-41ce-926f-543d2b190bad\" (UID: \"cd24f9bd-8033-41ce-926f-543d2b190bad\") " Jul 06 23:55:09.331611 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:09.331427 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd24f9bd-8033-41ce-926f-543d2b190bad-bundle\") pod \"cd24f9bd-8033-41ce-926f-543d2b190bad\" (UID: \"cd24f9bd-8033-41ce-926f-543d2b190bad\") " Jul 06 23:55:09.331611 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:09.331458 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqcvp\" (UniqueName: \"kubernetes.io/projected/cd24f9bd-8033-41ce-926f-543d2b190bad-kube-api-access-zqcvp\") pod \"cd24f9bd-8033-41ce-926f-543d2b190bad\" (UID: \"cd24f9bd-8033-41ce-926f-543d2b190bad\") " Jul 06 23:55:09.332353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:09.332327 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd24f9bd-8033-41ce-926f-543d2b190bad-bundle" (OuterVolumeSpecName: "bundle") pod "cd24f9bd-8033-41ce-926f-543d2b190bad" (UID: "cd24f9bd-8033-41ce-926f-543d2b190bad"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:55:09.333764 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:09.333734 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd24f9bd-8033-41ce-926f-543d2b190bad-kube-api-access-zqcvp" (OuterVolumeSpecName: "kube-api-access-zqcvp") pod "cd24f9bd-8033-41ce-926f-543d2b190bad" (UID: "cd24f9bd-8033-41ce-926f-543d2b190bad"). InnerVolumeSpecName "kube-api-access-zqcvp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:55:09.337414 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:09.337388 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd24f9bd-8033-41ce-926f-543d2b190bad-util" (OuterVolumeSpecName: "util") pod "cd24f9bd-8033-41ce-926f-543d2b190bad" (UID: "cd24f9bd-8033-41ce-926f-543d2b190bad"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:55:09.433176 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:09.433139 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd24f9bd-8033-41ce-926f-543d2b190bad-util\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:55:09.433176 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:09.433168 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd24f9bd-8033-41ce-926f-543d2b190bad-bundle\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:55:09.433176 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:09.433176 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqcvp\" (UniqueName: \"kubernetes.io/projected/cd24f9bd-8033-41ce-926f-543d2b190bad-kube-api-access-zqcvp\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:55:10.093680 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:10.093644 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7" event={"ID":"cd24f9bd-8033-41ce-926f-543d2b190bad","Type":"ContainerDied","Data":"d6a838e23485a5eaac4fdd1f5a8d8e96681ea813d73330d9c9a5cab76c54552b"} Jul 06 23:55:10.093680 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:10.093672 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdnz7" Jul 06 23:55:10.093900 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:10.093696 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d6a838e23485a5eaac4fdd1f5a8d8e96681ea813d73330d9c9a5cab76c54552b" Jul 06 23:55:20.694358 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.694323 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-xcv6b"] Jul 06 23:55:20.694782 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.694675 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd24f9bd-8033-41ce-926f-543d2b190bad" containerName="util" Jul 06 23:55:20.694782 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.694690 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd24f9bd-8033-41ce-926f-543d2b190bad" containerName="util" Jul 06 23:55:20.694782 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.694705 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd24f9bd-8033-41ce-926f-543d2b190bad" containerName="pull" Jul 06 23:55:20.694782 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.694713 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd24f9bd-8033-41ce-926f-543d2b190bad" containerName="pull" Jul 06 23:55:20.694782 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.694725 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd24f9bd-8033-41ce-926f-543d2b190bad" containerName="extract" Jul 06 23:55:20.694782 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.694731 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd24f9bd-8033-41ce-926f-543d2b190bad" containerName="extract" Jul 06 23:55:20.695007 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.694792 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd24f9bd-8033-41ce-926f-543d2b190bad" containerName="extract" Jul 06 23:55:20.698992 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.698969 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-xcv6b" Jul 06 23:55:20.701635 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.701611 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 23:55:20.701801 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.701778 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 23:55:20.702060 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.702045 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 23:55:20.702131 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.702115 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 23:55:20.702635 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.702619 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-cs9gm\"" Jul 06 23:55:20.712783 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.712764 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-xcv6b"] Jul 06 23:55:20.826943 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.826910 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0209cf45-9d4a-4f99-8cdb-29e120237003-manifests\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-xcv6b\" (UID: \"0209cf45-9d4a-4f99-8cdb-29e120237003\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-xcv6b" Jul 06 23:55:20.826943 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.826946 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0209cf45-9d4a-4f99-8cdb-29e120237003-tmp\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-xcv6b\" (UID: \"0209cf45-9d4a-4f99-8cdb-29e120237003\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-xcv6b" Jul 06 23:55:20.827171 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.826968 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0209cf45-9d4a-4f99-8cdb-29e120237003-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-xcv6b\" (UID: \"0209cf45-9d4a-4f99-8cdb-29e120237003\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-xcv6b" Jul 06 23:55:20.827171 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.826996 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75pvw\" (UniqueName: \"kubernetes.io/projected/0209cf45-9d4a-4f99-8cdb-29e120237003-kube-api-access-75pvw\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-xcv6b\" (UID: \"0209cf45-9d4a-4f99-8cdb-29e120237003\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-xcv6b" Jul 06 23:55:20.827171 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.827035 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0209cf45-9d4a-4f99-8cdb-29e120237003-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-xcv6b\" (UID: \"0209cf45-9d4a-4f99-8cdb-29e120237003\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-xcv6b" Jul 06 23:55:20.928428 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.928389 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0209cf45-9d4a-4f99-8cdb-29e120237003-manifests\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-xcv6b\" (UID: \"0209cf45-9d4a-4f99-8cdb-29e120237003\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-xcv6b" Jul 06 23:55:20.928428 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.928433 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0209cf45-9d4a-4f99-8cdb-29e120237003-tmp\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-xcv6b\" (UID: \"0209cf45-9d4a-4f99-8cdb-29e120237003\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-xcv6b" Jul 06 23:55:20.928688 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.928459 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0209cf45-9d4a-4f99-8cdb-29e120237003-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-xcv6b\" (UID: \"0209cf45-9d4a-4f99-8cdb-29e120237003\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-xcv6b" Jul 06 23:55:20.928688 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.928483 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75pvw\" (UniqueName: \"kubernetes.io/projected/0209cf45-9d4a-4f99-8cdb-29e120237003-kube-api-access-75pvw\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-xcv6b\" (UID: \"0209cf45-9d4a-4f99-8cdb-29e120237003\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-xcv6b" Jul 06 23:55:20.928688 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.928507 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0209cf45-9d4a-4f99-8cdb-29e120237003-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-xcv6b\" (UID: \"0209cf45-9d4a-4f99-8cdb-29e120237003\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-xcv6b" Jul 06 23:55:20.928970 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.928938 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0209cf45-9d4a-4f99-8cdb-29e120237003-manifests\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-xcv6b\" (UID: \"0209cf45-9d4a-4f99-8cdb-29e120237003\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-xcv6b" Jul 06 23:55:20.930816 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.930793 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0209cf45-9d4a-4f99-8cdb-29e120237003-tmp\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-xcv6b\" (UID: \"0209cf45-9d4a-4f99-8cdb-29e120237003\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-xcv6b" Jul 06 23:55:20.930999 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.930980 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0209cf45-9d4a-4f99-8cdb-29e120237003-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-xcv6b\" (UID: \"0209cf45-9d4a-4f99-8cdb-29e120237003\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-xcv6b" Jul 06 23:55:20.931121 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.931101 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0209cf45-9d4a-4f99-8cdb-29e120237003-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-xcv6b\" (UID: \"0209cf45-9d4a-4f99-8cdb-29e120237003\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-xcv6b" Jul 06 23:55:20.936807 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:20.936781 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75pvw\" (UniqueName: \"kubernetes.io/projected/0209cf45-9d4a-4f99-8cdb-29e120237003-kube-api-access-75pvw\") pod \"opendatahub-operator-controller-manager-5bcdcfc745-xcv6b\" (UID: \"0209cf45-9d4a-4f99-8cdb-29e120237003\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-xcv6b" Jul 06 23:55:21.009822 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:21.009740 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-xcv6b" Jul 06 23:55:21.205600 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:21.205570 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-xcv6b"] Jul 06 23:55:21.207793 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:55:21.207760 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0209cf45_9d4a_4f99_8cdb_29e120237003.slice/crio-fb98995836045384367bf99b88d901b842579bf07a376dac5b3775db883d1cfe WatchSource:0}: Error finding container fb98995836045384367bf99b88d901b842579bf07a376dac5b3775db883d1cfe: Status 404 returned error can't find the container with id fb98995836045384367bf99b88d901b842579bf07a376dac5b3775db883d1cfe Jul 06 23:55:22.155954 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:22.155910 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-xcv6b" event={"ID":"0209cf45-9d4a-4f99-8cdb-29e120237003","Type":"ContainerStarted","Data":"fb98995836045384367bf99b88d901b842579bf07a376dac5b3775db883d1cfe"} Jul 06 23:55:24.166598 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:24.166565 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-xcv6b" event={"ID":"0209cf45-9d4a-4f99-8cdb-29e120237003","Type":"ContainerStarted","Data":"5d589d3927592d299ffe4de6ca7d5d1717a2d7b1bbdf2699795237f25b15756b"} Jul 06 23:55:25.171939 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:25.171910 2553 generic.go:356] "Generic (PLEG): container finished" podID="0209cf45-9d4a-4f99-8cdb-29e120237003" containerID="5d589d3927592d299ffe4de6ca7d5d1717a2d7b1bbdf2699795237f25b15756b" exitCode=0 Jul 06 23:55:25.172353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:25.171974 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-xcv6b" event={"ID":"0209cf45-9d4a-4f99-8cdb-29e120237003","Type":"ContainerDied","Data":"5d589d3927592d299ffe4de6ca7d5d1717a2d7b1bbdf2699795237f25b15756b"} Jul 06 23:55:26.184241 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:26.184205 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-xcv6b" event={"ID":"0209cf45-9d4a-4f99-8cdb-29e120237003","Type":"ContainerStarted","Data":"7cb58ce0f3bd8a9c44ec5e08717b9b69c1d1e5e7e91654b70e38ffe503f0a243"} Jul 06 23:55:26.184713 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:26.184403 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-xcv6b" Jul 06 23:55:26.209916 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:26.209874 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-xcv6b" podStartSLOduration=1.970297784 podStartE2EDuration="6.209861467s" totalImagesPullingTime="3.238359177s" totalInitContainerRuntime="1.001204506s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:55:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:55:26.208267388 +0000 UTC m=+408.417996919" watchObservedRunningTime="2026-07-06 23:55:26.209861467 +0000 UTC m=+408.419590987" Jul 06 23:55:37.191063 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:37.191029 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5bcdcfc745-xcv6b" Jul 06 23:55:40.452190 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:40.452153 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5dc8c455ff-sbm28"] Jul 06 23:55:40.462349 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:40.462313 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5dc8c455ff-sbm28" Jul 06 23:55:40.468247 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:40.468218 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 06 23:55:40.475528 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:40.475497 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-vcxmg\"" Jul 06 23:55:40.475528 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:40.475505 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 23:55:40.476578 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:40.476560 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 06 23:55:40.476681 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:40.476662 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 23:55:40.489035 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:40.489003 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5dc8c455ff-sbm28"] Jul 06 23:55:40.601107 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:40.601069 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b94f94fd-f309-4495-86af-fb0eb3946b4f-tmp\") pod \"kube-auth-proxy-5dc8c455ff-sbm28\" (UID: \"b94f94fd-f309-4495-86af-fb0eb3946b4f\") " pod="openshift-ingress/kube-auth-proxy-5dc8c455ff-sbm28" Jul 06 23:55:40.601283 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:40.601115 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4z62\" (UniqueName: \"kubernetes.io/projected/b94f94fd-f309-4495-86af-fb0eb3946b4f-kube-api-access-g4z62\") pod \"kube-auth-proxy-5dc8c455ff-sbm28\" (UID: \"b94f94fd-f309-4495-86af-fb0eb3946b4f\") " pod="openshift-ingress/kube-auth-proxy-5dc8c455ff-sbm28" Jul 06 23:55:40.601283 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:40.601254 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b94f94fd-f309-4495-86af-fb0eb3946b4f-tls-certs\") pod \"kube-auth-proxy-5dc8c455ff-sbm28\" (UID: \"b94f94fd-f309-4495-86af-fb0eb3946b4f\") " pod="openshift-ingress/kube-auth-proxy-5dc8c455ff-sbm28" Jul 06 23:55:40.702665 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:40.702572 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b94f94fd-f309-4495-86af-fb0eb3946b4f-tls-certs\") pod \"kube-auth-proxy-5dc8c455ff-sbm28\" (UID: \"b94f94fd-f309-4495-86af-fb0eb3946b4f\") " pod="openshift-ingress/kube-auth-proxy-5dc8c455ff-sbm28" Jul 06 23:55:40.702665 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:40.702620 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b94f94fd-f309-4495-86af-fb0eb3946b4f-tmp\") pod \"kube-auth-proxy-5dc8c455ff-sbm28\" (UID: \"b94f94fd-f309-4495-86af-fb0eb3946b4f\") " pod="openshift-ingress/kube-auth-proxy-5dc8c455ff-sbm28" Jul 06 23:55:40.702665 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:40.702653 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4z62\" (UniqueName: \"kubernetes.io/projected/b94f94fd-f309-4495-86af-fb0eb3946b4f-kube-api-access-g4z62\") pod \"kube-auth-proxy-5dc8c455ff-sbm28\" (UID: \"b94f94fd-f309-4495-86af-fb0eb3946b4f\") " pod="openshift-ingress/kube-auth-proxy-5dc8c455ff-sbm28" Jul 06 23:55:40.705202 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:40.705174 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b94f94fd-f309-4495-86af-fb0eb3946b4f-tmp\") pod \"kube-auth-proxy-5dc8c455ff-sbm28\" (UID: \"b94f94fd-f309-4495-86af-fb0eb3946b4f\") " pod="openshift-ingress/kube-auth-proxy-5dc8c455ff-sbm28" Jul 06 23:55:40.705753 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:40.705719 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b94f94fd-f309-4495-86af-fb0eb3946b4f-tls-certs\") pod \"kube-auth-proxy-5dc8c455ff-sbm28\" (UID: \"b94f94fd-f309-4495-86af-fb0eb3946b4f\") " pod="openshift-ingress/kube-auth-proxy-5dc8c455ff-sbm28" Jul 06 23:55:40.740807 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:40.740776 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4z62\" (UniqueName: \"kubernetes.io/projected/b94f94fd-f309-4495-86af-fb0eb3946b4f-kube-api-access-g4z62\") pod \"kube-auth-proxy-5dc8c455ff-sbm28\" (UID: \"b94f94fd-f309-4495-86af-fb0eb3946b4f\") " pod="openshift-ingress/kube-auth-proxy-5dc8c455ff-sbm28" Jul 06 23:55:40.773110 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:40.773071 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5dc8c455ff-sbm28" Jul 06 23:55:40.922427 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:40.922379 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5dc8c455ff-sbm28"] Jul 06 23:55:40.925101 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:55:40.925068 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb94f94fd_f309_4495_86af_fb0eb3946b4f.slice/crio-7d226d9913ead1848989e64e2c4c6de4623665cb46f7797c6054b075cc0eae74 WatchSource:0}: Error finding container 7d226d9913ead1848989e64e2c4c6de4623665cb46f7797c6054b075cc0eae74: Status 404 returned error can't find the container with id 7d226d9913ead1848989e64e2c4c6de4623665cb46f7797c6054b075cc0eae74 Jul 06 23:55:41.256092 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:41.256056 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5dc8c455ff-sbm28" event={"ID":"b94f94fd-f309-4495-86af-fb0eb3946b4f","Type":"ContainerStarted","Data":"7d226d9913ead1848989e64e2c4c6de4623665cb46f7797c6054b075cc0eae74"} Jul 06 23:55:42.284766 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.284729 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs"] Jul 06 23:55:42.288176 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.288151 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" Jul 06 23:55:42.299167 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.298936 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 06 23:55:42.299167 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.298970 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 06 23:55:42.299167 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.298970 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 06 23:55:42.299167 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.299066 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 23:55:42.299593 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.299370 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-48vmd\"" Jul 06 23:55:42.308539 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.308510 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs"] Jul 06 23:55:42.416191 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.416121 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/870a9f76-5bb8-4a74-ad8a-6701fcd767df-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-cwvzs\" (UID: \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" Jul 06 23:55:42.416433 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.416215 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/870a9f76-5bb8-4a74-ad8a-6701fcd767df-istio-token\") pod \"istiod-openshift-gateway-7574459656-cwvzs\" (UID: \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" Jul 06 23:55:42.416433 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.416250 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/870a9f76-5bb8-4a74-ad8a-6701fcd767df-cacerts\") pod \"istiod-openshift-gateway-7574459656-cwvzs\" (UID: \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" Jul 06 23:55:42.416433 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.416280 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/870a9f76-5bb8-4a74-ad8a-6701fcd767df-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-cwvzs\" (UID: \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" Jul 06 23:55:42.416433 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.416372 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44gff\" (UniqueName: \"kubernetes.io/projected/870a9f76-5bb8-4a74-ad8a-6701fcd767df-kube-api-access-44gff\") pod \"istiod-openshift-gateway-7574459656-cwvzs\" (UID: \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" Jul 06 23:55:42.416433 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.416417 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/870a9f76-5bb8-4a74-ad8a-6701fcd767df-local-certs\") pod \"istiod-openshift-gateway-7574459656-cwvzs\" (UID: \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" Jul 06 23:55:42.416697 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.416521 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/870a9f76-5bb8-4a74-ad8a-6701fcd767df-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-cwvzs\" (UID: \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" Jul 06 23:55:42.517914 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.517863 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/870a9f76-5bb8-4a74-ad8a-6701fcd767df-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-cwvzs\" (UID: \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" Jul 06 23:55:42.518097 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.517942 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/870a9f76-5bb8-4a74-ad8a-6701fcd767df-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-cwvzs\" (UID: \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" Jul 06 23:55:42.518097 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.517979 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/870a9f76-5bb8-4a74-ad8a-6701fcd767df-istio-token\") pod \"istiod-openshift-gateway-7574459656-cwvzs\" (UID: \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" Jul 06 23:55:42.518097 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.518013 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/870a9f76-5bb8-4a74-ad8a-6701fcd767df-cacerts\") pod \"istiod-openshift-gateway-7574459656-cwvzs\" (UID: \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" Jul 06 23:55:42.518097 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.518043 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/870a9f76-5bb8-4a74-ad8a-6701fcd767df-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-cwvzs\" (UID: \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" Jul 06 23:55:42.518097 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.518075 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44gff\" (UniqueName: \"kubernetes.io/projected/870a9f76-5bb8-4a74-ad8a-6701fcd767df-kube-api-access-44gff\") pod \"istiod-openshift-gateway-7574459656-cwvzs\" (UID: \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" Jul 06 23:55:42.518273 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.518097 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/870a9f76-5bb8-4a74-ad8a-6701fcd767df-local-certs\") pod \"istiod-openshift-gateway-7574459656-cwvzs\" (UID: \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" Jul 06 23:55:42.518626 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.518597 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/870a9f76-5bb8-4a74-ad8a-6701fcd767df-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-cwvzs\" (UID: \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" Jul 06 23:55:42.521696 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.521221 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/870a9f76-5bb8-4a74-ad8a-6701fcd767df-cacerts\") pod \"istiod-openshift-gateway-7574459656-cwvzs\" (UID: \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" Jul 06 23:55:42.521696 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.521368 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/870a9f76-5bb8-4a74-ad8a-6701fcd767df-local-certs\") pod \"istiod-openshift-gateway-7574459656-cwvzs\" (UID: \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" Jul 06 23:55:42.521696 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.521648 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/870a9f76-5bb8-4a74-ad8a-6701fcd767df-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-cwvzs\" (UID: \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" Jul 06 23:55:42.521966 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.521757 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/870a9f76-5bb8-4a74-ad8a-6701fcd767df-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-cwvzs\" (UID: \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" Jul 06 23:55:42.537025 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.536939 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/870a9f76-5bb8-4a74-ad8a-6701fcd767df-istio-token\") pod \"istiod-openshift-gateway-7574459656-cwvzs\" (UID: \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" Jul 06 23:55:42.537422 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.537374 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44gff\" (UniqueName: \"kubernetes.io/projected/870a9f76-5bb8-4a74-ad8a-6701fcd767df-kube-api-access-44gff\") pod \"istiod-openshift-gateway-7574459656-cwvzs\" (UID: \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" Jul 06 23:55:42.610251 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:42.610212 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" Jul 06 23:55:43.397092 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:43.397051 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs"] Jul 06 23:55:43.399977 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:55:43.399934 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod870a9f76_5bb8_4a74_ad8a_6701fcd767df.slice/crio-0cd70cdeb89f4a19baf8fe1b0ee74f6af89f40b17f3dc606cec9e48fb8c0de03 WatchSource:0}: Error finding container 0cd70cdeb89f4a19baf8fe1b0ee74f6af89f40b17f3dc606cec9e48fb8c0de03: Status 404 returned error can't find the container with id 0cd70cdeb89f4a19baf8fe1b0ee74f6af89f40b17f3dc606cec9e48fb8c0de03 Jul 06 23:55:44.275778 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:44.275732 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" event={"ID":"870a9f76-5bb8-4a74-ad8a-6701fcd767df","Type":"ContainerStarted","Data":"0cd70cdeb89f4a19baf8fe1b0ee74f6af89f40b17f3dc606cec9e48fb8c0de03"} Jul 06 23:55:44.278037 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:44.278004 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5dc8c455ff-sbm28" event={"ID":"b94f94fd-f309-4495-86af-fb0eb3946b4f","Type":"ContainerStarted","Data":"8a0ef5ab168c472703e69797db728b106b4d265a5dc13fb98a59c8089fe1efff"} Jul 06 23:55:44.298829 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:44.298765 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5dc8c455ff-sbm28" podStartSLOduration=1.9344819659999999 podStartE2EDuration="4.298744053s" totalImagesPullingTime="2.364262087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:55:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:55:44.297770737 +0000 UTC m=+426.507500282" watchObservedRunningTime="2026-07-06 23:55:44.298744053 +0000 UTC m=+426.508473572" Jul 06 23:55:45.864977 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:45.864935 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 23:55:45.865392 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:45.865008 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 23:55:46.292264 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:46.292223 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" event={"ID":"870a9f76-5bb8-4a74-ad8a-6701fcd767df","Type":"ContainerStarted","Data":"a38f3e3b1df10ff3fe1df9f77b7e3c48919b1a747492cb1487f9e73fd9d2b840"} Jul 06 23:55:46.313805 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:46.313742 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" podStartSLOduration=1.8513399160000001 podStartE2EDuration="4.313718024s" totalImagesPullingTime="2.462378108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:55:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:55:46.312495873 +0000 UTC m=+428.522225422" watchObservedRunningTime="2026-07-06 23:55:46.313718024 +0000 UTC m=+428.523447547" Jul 06 23:55:47.297315 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.297263 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" Jul 06 23:55:47.299615 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.299583 2553 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-cwvzs container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 23:55:47.299764 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.299646 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" podUID="870a9f76-5bb8-4a74-ad8a-6701fcd767df" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 23:55:47.645833 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.645797 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9"] Jul 06 23:55:47.649496 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.649463 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:47.652620 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.652598 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-xf22d\"" Jul 06 23:55:47.672214 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.672181 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9"] Jul 06 23:55:47.764938 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.764887 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/379fe0e0-dc84-4b69-a99d-dcf861e32ce6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678sppw9\" (UID: \"379fe0e0-dc84-4b69-a99d-dcf861e32ce6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:47.765124 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.764951 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/379fe0e0-dc84-4b69-a99d-dcf861e32ce6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678sppw9\" (UID: \"379fe0e0-dc84-4b69-a99d-dcf861e32ce6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:47.765124 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.764991 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/379fe0e0-dc84-4b69-a99d-dcf861e32ce6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678sppw9\" (UID: \"379fe0e0-dc84-4b69-a99d-dcf861e32ce6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:47.765124 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.765072 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/379fe0e0-dc84-4b69-a99d-dcf861e32ce6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678sppw9\" (UID: \"379fe0e0-dc84-4b69-a99d-dcf861e32ce6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:47.765124 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.765097 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l69hn\" (UniqueName: \"kubernetes.io/projected/379fe0e0-dc84-4b69-a99d-dcf861e32ce6-kube-api-access-l69hn\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678sppw9\" (UID: \"379fe0e0-dc84-4b69-a99d-dcf861e32ce6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:47.765345 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.765132 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/379fe0e0-dc84-4b69-a99d-dcf861e32ce6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678sppw9\" (UID: \"379fe0e0-dc84-4b69-a99d-dcf861e32ce6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:47.765345 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.765156 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/379fe0e0-dc84-4b69-a99d-dcf861e32ce6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678sppw9\" (UID: \"379fe0e0-dc84-4b69-a99d-dcf861e32ce6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:47.765345 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.765225 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/379fe0e0-dc84-4b69-a99d-dcf861e32ce6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678sppw9\" (UID: \"379fe0e0-dc84-4b69-a99d-dcf861e32ce6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:47.765345 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.765285 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/379fe0e0-dc84-4b69-a99d-dcf861e32ce6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678sppw9\" (UID: \"379fe0e0-dc84-4b69-a99d-dcf861e32ce6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:47.866417 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.866380 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/379fe0e0-dc84-4b69-a99d-dcf861e32ce6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678sppw9\" (UID: \"379fe0e0-dc84-4b69-a99d-dcf861e32ce6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:47.866417 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.866420 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l69hn\" (UniqueName: \"kubernetes.io/projected/379fe0e0-dc84-4b69-a99d-dcf861e32ce6-kube-api-access-l69hn\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678sppw9\" (UID: \"379fe0e0-dc84-4b69-a99d-dcf861e32ce6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:47.866647 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.866446 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/379fe0e0-dc84-4b69-a99d-dcf861e32ce6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678sppw9\" (UID: \"379fe0e0-dc84-4b69-a99d-dcf861e32ce6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:47.866647 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.866522 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/379fe0e0-dc84-4b69-a99d-dcf861e32ce6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678sppw9\" (UID: \"379fe0e0-dc84-4b69-a99d-dcf861e32ce6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:47.866647 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.866582 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/379fe0e0-dc84-4b69-a99d-dcf861e32ce6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678sppw9\" (UID: \"379fe0e0-dc84-4b69-a99d-dcf861e32ce6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:47.866647 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.866621 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/379fe0e0-dc84-4b69-a99d-dcf861e32ce6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678sppw9\" (UID: \"379fe0e0-dc84-4b69-a99d-dcf861e32ce6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:47.866882 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.866654 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/379fe0e0-dc84-4b69-a99d-dcf861e32ce6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678sppw9\" (UID: \"379fe0e0-dc84-4b69-a99d-dcf861e32ce6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:47.866882 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.866695 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/379fe0e0-dc84-4b69-a99d-dcf861e32ce6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678sppw9\" (UID: \"379fe0e0-dc84-4b69-a99d-dcf861e32ce6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:47.866882 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.866734 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/379fe0e0-dc84-4b69-a99d-dcf861e32ce6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678sppw9\" (UID: \"379fe0e0-dc84-4b69-a99d-dcf861e32ce6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:47.866882 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.866864 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/379fe0e0-dc84-4b69-a99d-dcf861e32ce6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678sppw9\" (UID: \"379fe0e0-dc84-4b69-a99d-dcf861e32ce6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:47.867093 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.866882 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/379fe0e0-dc84-4b69-a99d-dcf861e32ce6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678sppw9\" (UID: \"379fe0e0-dc84-4b69-a99d-dcf861e32ce6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:47.867093 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.867066 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/379fe0e0-dc84-4b69-a99d-dcf861e32ce6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678sppw9\" (UID: \"379fe0e0-dc84-4b69-a99d-dcf861e32ce6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:47.867496 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.867469 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/379fe0e0-dc84-4b69-a99d-dcf861e32ce6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678sppw9\" (UID: \"379fe0e0-dc84-4b69-a99d-dcf861e32ce6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:47.867496 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.867487 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/379fe0e0-dc84-4b69-a99d-dcf861e32ce6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678sppw9\" (UID: \"379fe0e0-dc84-4b69-a99d-dcf861e32ce6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:47.869247 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.869203 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/379fe0e0-dc84-4b69-a99d-dcf861e32ce6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678sppw9\" (UID: \"379fe0e0-dc84-4b69-a99d-dcf861e32ce6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:47.869517 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.869491 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/379fe0e0-dc84-4b69-a99d-dcf861e32ce6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678sppw9\" (UID: \"379fe0e0-dc84-4b69-a99d-dcf861e32ce6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:47.878524 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.878495 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/379fe0e0-dc84-4b69-a99d-dcf861e32ce6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678sppw9\" (UID: \"379fe0e0-dc84-4b69-a99d-dcf861e32ce6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:47.878669 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.878529 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l69hn\" (UniqueName: \"kubernetes.io/projected/379fe0e0-dc84-4b69-a99d-dcf861e32ce6-kube-api-access-l69hn\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678sppw9\" (UID: \"379fe0e0-dc84-4b69-a99d-dcf861e32ce6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:47.960574 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:47.960477 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:48.133921 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:48.133878 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9"] Jul 06 23:55:48.140362 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:55:48.140319 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod379fe0e0_dc84_4b69_a99d_dcf861e32ce6.slice/crio-3329b957b0ba94583f976094f5cab752a9468af1b7804d05a51b672f7f7e7aae WatchSource:0}: Error finding container 3329b957b0ba94583f976094f5cab752a9468af1b7804d05a51b672f7f7e7aae: Status 404 returned error can't find the container with id 3329b957b0ba94583f976094f5cab752a9468af1b7804d05a51b672f7f7e7aae Jul 06 23:55:48.311458 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:48.311373 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" event={"ID":"379fe0e0-dc84-4b69-a99d-dcf861e32ce6","Type":"ContainerStarted","Data":"3329b957b0ba94583f976094f5cab752a9468af1b7804d05a51b672f7f7e7aae"} Jul 06 23:55:48.311900 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:48.311881 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" Jul 06 23:55:50.633938 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:50.633772 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 23:55:50.633938 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:50.633894 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 23:55:50.634626 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:50.633942 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 23:55:51.330335 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:51.330276 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" event={"ID":"379fe0e0-dc84-4b69-a99d-dcf861e32ce6","Type":"ContainerStarted","Data":"ae95ad1c204fe41035b085efe0417046747218edd1faaae883c3c9ec5f93e2e4"} Jul 06 23:55:51.362016 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:51.361958 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" podStartSLOduration=1.871542132 podStartE2EDuration="4.361940243s" totalImagesPullingTime="2.490398111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:55:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:55:51.359475494 +0000 UTC m=+433.569205037" watchObservedRunningTime="2026-07-06 23:55:51.361940243 +0000 UTC m=+433.571669764" Jul 06 23:55:51.961073 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:51.961035 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:51.967260 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:51.967227 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:52.335915 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:52.335833 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:52.337371 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:52.337350 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678sppw9" Jul 06 23:55:52.403169 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:52.403134 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-szrs8"] Jul 06 23:55:52.407188 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:52.407156 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-szrs8" Jul 06 23:55:52.409844 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:52.409814 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 06 23:55:52.410041 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:52.409907 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-l7zz8\"" Jul 06 23:55:52.411752 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:52.411725 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq447\" (UniqueName: \"kubernetes.io/projected/6df592ff-ca58-4349-9af2-0d2175c2e995-kube-api-access-bq447\") pod \"kserve-controller-manager-d6f4b65b8-szrs8\" (UID: \"6df592ff-ca58-4349-9af2-0d2175c2e995\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-szrs8" Jul 06 23:55:52.412173 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:52.412140 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6df592ff-ca58-4349-9af2-0d2175c2e995-cert\") pod \"kserve-controller-manager-d6f4b65b8-szrs8\" (UID: \"6df592ff-ca58-4349-9af2-0d2175c2e995\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-szrs8" Jul 06 23:55:52.415615 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:52.414992 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bwhj6"] Jul 06 23:55:52.418890 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:52.418865 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bwhj6" Jul 06 23:55:52.421434 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:52.421412 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 06 23:55:52.422497 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:52.422477 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-2rs9f\"" Jul 06 23:55:52.423138 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:52.423114 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-szrs8"] Jul 06 23:55:52.437172 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:52.437140 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bwhj6"] Jul 06 23:55:52.514167 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:52.514131 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq447\" (UniqueName: \"kubernetes.io/projected/6df592ff-ca58-4349-9af2-0d2175c2e995-kube-api-access-bq447\") pod \"kserve-controller-manager-d6f4b65b8-szrs8\" (UID: \"6df592ff-ca58-4349-9af2-0d2175c2e995\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-szrs8" Jul 06 23:55:52.514373 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:52.514218 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pp45\" (UniqueName: \"kubernetes.io/projected/e4805476-0013-484a-b046-999b6308296f-kube-api-access-5pp45\") pod \"llmisvc-controller-manager-777c6bd7d5-bwhj6\" (UID: \"e4805476-0013-484a-b046-999b6308296f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bwhj6" Jul 06 23:55:52.514373 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:52.514264 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e4805476-0013-484a-b046-999b6308296f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bwhj6\" (UID: \"e4805476-0013-484a-b046-999b6308296f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bwhj6" Jul 06 23:55:52.514373 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:52.514345 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6df592ff-ca58-4349-9af2-0d2175c2e995-cert\") pod \"kserve-controller-manager-d6f4b65b8-szrs8\" (UID: \"6df592ff-ca58-4349-9af2-0d2175c2e995\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-szrs8" Jul 06 23:55:52.514493 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:55:52.514478 2553 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 06 23:55:52.514590 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:55:52.514579 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6df592ff-ca58-4349-9af2-0d2175c2e995-cert podName:6df592ff-ca58-4349-9af2-0d2175c2e995 nodeName:}" failed. No retries permitted until 2026-07-06 23:55:53.014556198 +0000 UTC m=+435.224285697 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6df592ff-ca58-4349-9af2-0d2175c2e995-cert") pod "kserve-controller-manager-d6f4b65b8-szrs8" (UID: "6df592ff-ca58-4349-9af2-0d2175c2e995") : secret "kserve-webhook-server-cert" not found Jul 06 23:55:52.524925 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:52.524896 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq447\" (UniqueName: \"kubernetes.io/projected/6df592ff-ca58-4349-9af2-0d2175c2e995-kube-api-access-bq447\") pod \"kserve-controller-manager-d6f4b65b8-szrs8\" (UID: \"6df592ff-ca58-4349-9af2-0d2175c2e995\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-szrs8" Jul 06 23:55:52.615730 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:52.615633 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pp45\" (UniqueName: \"kubernetes.io/projected/e4805476-0013-484a-b046-999b6308296f-kube-api-access-5pp45\") pod \"llmisvc-controller-manager-777c6bd7d5-bwhj6\" (UID: \"e4805476-0013-484a-b046-999b6308296f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bwhj6" Jul 06 23:55:52.615730 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:52.615690 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e4805476-0013-484a-b046-999b6308296f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bwhj6\" (UID: \"e4805476-0013-484a-b046-999b6308296f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bwhj6" Jul 06 23:55:52.615945 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:55:52.615777 2553 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 06 23:55:52.615945 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:55:52.615841 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4805476-0013-484a-b046-999b6308296f-cert podName:e4805476-0013-484a-b046-999b6308296f nodeName:}" failed. No retries permitted until 2026-07-06 23:55:53.115826841 +0000 UTC m=+435.325556343 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e4805476-0013-484a-b046-999b6308296f-cert") pod "llmisvc-controller-manager-777c6bd7d5-bwhj6" (UID: "e4805476-0013-484a-b046-999b6308296f") : secret "llmisvc-webhook-server-cert" not found Jul 06 23:55:52.625332 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:52.625277 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pp45\" (UniqueName: \"kubernetes.io/projected/e4805476-0013-484a-b046-999b6308296f-kube-api-access-5pp45\") pod \"llmisvc-controller-manager-777c6bd7d5-bwhj6\" (UID: \"e4805476-0013-484a-b046-999b6308296f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bwhj6" Jul 06 23:55:53.018993 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:53.018954 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6df592ff-ca58-4349-9af2-0d2175c2e995-cert\") pod \"kserve-controller-manager-d6f4b65b8-szrs8\" (UID: \"6df592ff-ca58-4349-9af2-0d2175c2e995\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-szrs8" Jul 06 23:55:53.021507 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:53.021483 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6df592ff-ca58-4349-9af2-0d2175c2e995-cert\") pod \"kserve-controller-manager-d6f4b65b8-szrs8\" (UID: \"6df592ff-ca58-4349-9af2-0d2175c2e995\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-szrs8" Jul 06 23:55:53.120353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:53.120278 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e4805476-0013-484a-b046-999b6308296f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bwhj6\" (UID: \"e4805476-0013-484a-b046-999b6308296f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bwhj6" Jul 06 23:55:53.123055 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:53.123033 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e4805476-0013-484a-b046-999b6308296f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bwhj6\" (UID: \"e4805476-0013-484a-b046-999b6308296f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bwhj6" Jul 06 23:55:53.321346 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:53.321216 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-szrs8" Jul 06 23:55:53.331551 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:53.331522 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bwhj6" Jul 06 23:55:53.540366 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:53.540287 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bwhj6"] Jul 06 23:55:53.543967 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:55:53.543941 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode4805476_0013_484a_b046_999b6308296f.slice/crio-ba46383d5adae62a6d077db0bd202d816753c3e620ef77817e24cdebcb71d170 WatchSource:0}: Error finding container ba46383d5adae62a6d077db0bd202d816753c3e620ef77817e24cdebcb71d170: Status 404 returned error can't find the container with id ba46383d5adae62a6d077db0bd202d816753c3e620ef77817e24cdebcb71d170 Jul 06 23:55:53.704334 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:53.704290 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-szrs8"] Jul 06 23:55:53.706515 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:55:53.706483 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6df592ff_ca58_4349_9af2_0d2175c2e995.slice/crio-349ed2924e6d7deea8fda0ab9df89ca53004a6c75111913725cf40f088e30449 WatchSource:0}: Error finding container 349ed2924e6d7deea8fda0ab9df89ca53004a6c75111913725cf40f088e30449: Status 404 returned error can't find the container with id 349ed2924e6d7deea8fda0ab9df89ca53004a6c75111913725cf40f088e30449 Jul 06 23:55:54.346739 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:54.346699 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-szrs8" event={"ID":"6df592ff-ca58-4349-9af2-0d2175c2e995","Type":"ContainerStarted","Data":"349ed2924e6d7deea8fda0ab9df89ca53004a6c75111913725cf40f088e30449"} Jul 06 23:55:54.348183 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:54.348154 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bwhj6" event={"ID":"e4805476-0013-484a-b046-999b6308296f","Type":"ContainerStarted","Data":"ba46383d5adae62a6d077db0bd202d816753c3e620ef77817e24cdebcb71d170"} Jul 06 23:55:54.570154 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:54.570112 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs"] Jul 06 23:55:54.570667 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:54.570378 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" podUID="870a9f76-5bb8-4a74-ad8a-6701fcd767df" containerName="discovery" containerID="cri-o://a38f3e3b1df10ff3fe1df9f77b7e3c48919b1a747492cb1487f9e73fd9d2b840" gracePeriod=30 Jul 06 23:55:54.963228 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:54.963113 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" Jul 06 23:55:55.044933 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.044895 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/870a9f76-5bb8-4a74-ad8a-6701fcd767df-cacerts\") pod \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\" (UID: \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\") " Jul 06 23:55:55.045113 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.044946 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/870a9f76-5bb8-4a74-ad8a-6701fcd767df-istio-csr-ca-configmap\") pod \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\" (UID: \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\") " Jul 06 23:55:55.045113 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.044980 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/870a9f76-5bb8-4a74-ad8a-6701fcd767df-local-certs\") pod \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\" (UID: \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\") " Jul 06 23:55:55.045113 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.045023 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44gff\" (UniqueName: \"kubernetes.io/projected/870a9f76-5bb8-4a74-ad8a-6701fcd767df-kube-api-access-44gff\") pod \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\" (UID: \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\") " Jul 06 23:55:55.045723 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.045689 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/870a9f76-5bb8-4a74-ad8a-6701fcd767df-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "870a9f76-5bb8-4a74-ad8a-6701fcd767df" (UID: "870a9f76-5bb8-4a74-ad8a-6701fcd767df"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:55:55.047793 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.047756 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/870a9f76-5bb8-4a74-ad8a-6701fcd767df-local-certs" (OuterVolumeSpecName: "local-certs") pod "870a9f76-5bb8-4a74-ad8a-6701fcd767df" (UID: "870a9f76-5bb8-4a74-ad8a-6701fcd767df"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:55:55.047793 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.047762 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/870a9f76-5bb8-4a74-ad8a-6701fcd767df-kube-api-access-44gff" (OuterVolumeSpecName: "kube-api-access-44gff") pod "870a9f76-5bb8-4a74-ad8a-6701fcd767df" (UID: "870a9f76-5bb8-4a74-ad8a-6701fcd767df"). InnerVolumeSpecName "kube-api-access-44gff". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:55:55.048199 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.048172 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/870a9f76-5bb8-4a74-ad8a-6701fcd767df-cacerts" (OuterVolumeSpecName: "cacerts") pod "870a9f76-5bb8-4a74-ad8a-6701fcd767df" (UID: "870a9f76-5bb8-4a74-ad8a-6701fcd767df"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:55:55.145913 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.145873 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/870a9f76-5bb8-4a74-ad8a-6701fcd767df-istio-kubeconfig\") pod \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\" (UID: \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\") " Jul 06 23:55:55.145913 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.145950 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/870a9f76-5bb8-4a74-ad8a-6701fcd767df-istio-token\") pod \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\" (UID: \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\") " Jul 06 23:55:55.145913 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.145978 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/870a9f76-5bb8-4a74-ad8a-6701fcd767df-istio-csr-dns-cert\") pod \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\" (UID: \"870a9f76-5bb8-4a74-ad8a-6701fcd767df\") " Jul 06 23:55:55.146333 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.146274 2553 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/870a9f76-5bb8-4a74-ad8a-6701fcd767df-cacerts\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:55:55.146333 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.146325 2553 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/870a9f76-5bb8-4a74-ad8a-6701fcd767df-istio-csr-ca-configmap\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:55:55.146467 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.146341 2553 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/870a9f76-5bb8-4a74-ad8a-6701fcd767df-local-certs\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:55:55.146467 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.146355 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44gff\" (UniqueName: \"kubernetes.io/projected/870a9f76-5bb8-4a74-ad8a-6701fcd767df-kube-api-access-44gff\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:55:55.148539 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.148501 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/870a9f76-5bb8-4a74-ad8a-6701fcd767df-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "870a9f76-5bb8-4a74-ad8a-6701fcd767df" (UID: "870a9f76-5bb8-4a74-ad8a-6701fcd767df"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:55:55.148680 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.148572 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/870a9f76-5bb8-4a74-ad8a-6701fcd767df-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "870a9f76-5bb8-4a74-ad8a-6701fcd767df" (UID: "870a9f76-5bb8-4a74-ad8a-6701fcd767df"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:55:55.149534 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.149503 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/870a9f76-5bb8-4a74-ad8a-6701fcd767df-istio-token" (OuterVolumeSpecName: "istio-token") pod "870a9f76-5bb8-4a74-ad8a-6701fcd767df" (UID: "870a9f76-5bb8-4a74-ad8a-6701fcd767df"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:55:55.247665 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.247573 2553 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/870a9f76-5bb8-4a74-ad8a-6701fcd767df-istio-kubeconfig\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:55:55.247665 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.247605 2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/870a9f76-5bb8-4a74-ad8a-6701fcd767df-istio-token\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:55:55.247665 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.247619 2553 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/870a9f76-5bb8-4a74-ad8a-6701fcd767df-istio-csr-dns-cert\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:55:55.354899 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.354858 2553 generic.go:356] "Generic (PLEG): container finished" podID="870a9f76-5bb8-4a74-ad8a-6701fcd767df" containerID="a38f3e3b1df10ff3fe1df9f77b7e3c48919b1a747492cb1487f9e73fd9d2b840" exitCode=0 Jul 06 23:55:55.354899 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.354901 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" Jul 06 23:55:55.355464 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.354947 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" event={"ID":"870a9f76-5bb8-4a74-ad8a-6701fcd767df","Type":"ContainerDied","Data":"a38f3e3b1df10ff3fe1df9f77b7e3c48919b1a747492cb1487f9e73fd9d2b840"} Jul 06 23:55:55.355464 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.355003 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs" event={"ID":"870a9f76-5bb8-4a74-ad8a-6701fcd767df","Type":"ContainerDied","Data":"0cd70cdeb89f4a19baf8fe1b0ee74f6af89f40b17f3dc606cec9e48fb8c0de03"} Jul 06 23:55:55.355464 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.355030 2553 scope.go:117] "RemoveContainer" containerID="a38f3e3b1df10ff3fe1df9f77b7e3c48919b1a747492cb1487f9e73fd9d2b840" Jul 06 23:55:55.365638 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.365611 2553 scope.go:117] "RemoveContainer" containerID="a38f3e3b1df10ff3fe1df9f77b7e3c48919b1a747492cb1487f9e73fd9d2b840" Jul 06 23:55:55.365958 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:55:55.365938 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a38f3e3b1df10ff3fe1df9f77b7e3c48919b1a747492cb1487f9e73fd9d2b840\": container with ID starting with a38f3e3b1df10ff3fe1df9f77b7e3c48919b1a747492cb1487f9e73fd9d2b840 not found: ID does not exist" containerID="a38f3e3b1df10ff3fe1df9f77b7e3c48919b1a747492cb1487f9e73fd9d2b840" Jul 06 23:55:55.366018 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.365967 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a38f3e3b1df10ff3fe1df9f77b7e3c48919b1a747492cb1487f9e73fd9d2b840"} err="failed to get container status \"a38f3e3b1df10ff3fe1df9f77b7e3c48919b1a747492cb1487f9e73fd9d2b840\": rpc error: code = NotFound desc = could not find container \"a38f3e3b1df10ff3fe1df9f77b7e3c48919b1a747492cb1487f9e73fd9d2b840\": container with ID starting with a38f3e3b1df10ff3fe1df9f77b7e3c48919b1a747492cb1487f9e73fd9d2b840 not found: ID does not exist" Jul 06 23:55:55.383489 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.383445 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs"] Jul 06 23:55:55.388991 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:55.388958 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cwvzs"] Jul 06 23:55:56.303916 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:56.303877 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="870a9f76-5bb8-4a74-ad8a-6701fcd767df" path="/var/lib/kubelet/pods/870a9f76-5bb8-4a74-ad8a-6701fcd767df/volumes" Jul 06 23:55:57.370733 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:57.370679 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-szrs8" event={"ID":"6df592ff-ca58-4349-9af2-0d2175c2e995","Type":"ContainerStarted","Data":"e268c8a2bfb30f9bc7d98b1673ceb8c3c2bd0e6f5663cf13bb1da06b37c65353"} Jul 06 23:55:57.371382 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:57.370864 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-szrs8" Jul 06 23:55:57.372759 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:57.372714 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bwhj6" event={"ID":"e4805476-0013-484a-b046-999b6308296f","Type":"ContainerStarted","Data":"f3e06032eba6eec64f7450af8def0c532852eb3383004432376957d82e258056"} Jul 06 23:55:57.372936 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:57.372838 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bwhj6" Jul 06 23:55:57.389843 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:57.389783 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-szrs8" podStartSLOduration=2.288195395 podStartE2EDuration="5.389766823s" totalImagesPullingTime="3.101571428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:55:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:55:57.388571899 +0000 UTC m=+439.598301440" watchObservedRunningTime="2026-07-06 23:55:57.389766823 +0000 UTC m=+439.599496344" Jul 06 23:55:57.407029 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:55:57.406975 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bwhj6" podStartSLOduration=2.090325983 podStartE2EDuration="5.406958485s" totalImagesPullingTime="3.316632502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:55:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:55:57.406228554 +0000 UTC m=+439.615958076" watchObservedRunningTime="2026-07-06 23:55:57.406958485 +0000 UTC m=+439.616688026" Jul 06 23:56:28.378981 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:28.378944 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bwhj6" Jul 06 23:56:28.381965 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:28.381947 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-szrs8" Jul 06 23:56:41.098359 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:41.098316 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r297p"] Jul 06 23:56:41.098860 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:41.098783 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="870a9f76-5bb8-4a74-ad8a-6701fcd767df" containerName="discovery" Jul 06 23:56:41.098860 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:41.098802 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="870a9f76-5bb8-4a74-ad8a-6701fcd767df" containerName="discovery" Jul 06 23:56:41.098978 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:41.098908 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="870a9f76-5bb8-4a74-ad8a-6701fcd767df" containerName="discovery" Jul 06 23:56:41.103481 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:41.103453 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r297p" Jul 06 23:56:41.106286 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:41.106254 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 23:56:41.107391 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:41.107352 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-9lc5x\"" Jul 06 23:56:41.107391 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:41.107355 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 23:56:41.110376 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:41.110349 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r297p"] Jul 06 23:56:41.248994 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:41.248956 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7f9w\" (UniqueName: \"kubernetes.io/projected/c20a4f0c-44fe-40ca-8461-57cca8b4c534-kube-api-access-c7f9w\") pod \"kuadrant-operator-catalog-r297p\" (UID: \"c20a4f0c-44fe-40ca-8461-57cca8b4c534\") " pod="kuadrant-system/kuadrant-operator-catalog-r297p" Jul 06 23:56:41.349653 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:41.349552 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7f9w\" (UniqueName: \"kubernetes.io/projected/c20a4f0c-44fe-40ca-8461-57cca8b4c534-kube-api-access-c7f9w\") pod \"kuadrant-operator-catalog-r297p\" (UID: \"c20a4f0c-44fe-40ca-8461-57cca8b4c534\") " pod="kuadrant-system/kuadrant-operator-catalog-r297p" Jul 06 23:56:41.359958 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:41.359923 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7f9w\" (UniqueName: \"kubernetes.io/projected/c20a4f0c-44fe-40ca-8461-57cca8b4c534-kube-api-access-c7f9w\") pod \"kuadrant-operator-catalog-r297p\" (UID: \"c20a4f0c-44fe-40ca-8461-57cca8b4c534\") " pod="kuadrant-system/kuadrant-operator-catalog-r297p" Jul 06 23:56:41.415698 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:41.415666 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r297p" Jul 06 23:56:41.477796 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:41.477762 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r297p"] Jul 06 23:56:41.556652 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:41.556626 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r297p"] Jul 06 23:56:41.559404 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:56:41.559375 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc20a4f0c_44fe_40ca_8461_57cca8b4c534.slice/crio-c4b0ddd9ac81893c8b0881c28392b78726333d57bdc2b039b4ee99f496084837 WatchSource:0}: Error finding container c4b0ddd9ac81893c8b0881c28392b78726333d57bdc2b039b4ee99f496084837: Status 404 returned error can't find the container with id c4b0ddd9ac81893c8b0881c28392b78726333d57bdc2b039b4ee99f496084837 Jul 06 23:56:41.583243 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:41.583203 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r297p" event={"ID":"c20a4f0c-44fe-40ca-8461-57cca8b4c534","Type":"ContainerStarted","Data":"c4b0ddd9ac81893c8b0881c28392b78726333d57bdc2b039b4ee99f496084837"} Jul 06 23:56:41.677444 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:41.677403 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pmjrz"] Jul 06 23:56:41.681812 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:41.681791 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pmjrz" Jul 06 23:56:41.689972 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:41.689937 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pmjrz"] Jul 06 23:56:41.854518 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:41.854482 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb4df\" (UniqueName: \"kubernetes.io/projected/8708739e-7b51-4394-9e97-89a8c1b87997-kube-api-access-rb4df\") pod \"kuadrant-operator-catalog-pmjrz\" (UID: \"8708739e-7b51-4394-9e97-89a8c1b87997\") " pod="kuadrant-system/kuadrant-operator-catalog-pmjrz" Jul 06 23:56:41.955223 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:41.955127 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb4df\" (UniqueName: \"kubernetes.io/projected/8708739e-7b51-4394-9e97-89a8c1b87997-kube-api-access-rb4df\") pod \"kuadrant-operator-catalog-pmjrz\" (UID: \"8708739e-7b51-4394-9e97-89a8c1b87997\") " pod="kuadrant-system/kuadrant-operator-catalog-pmjrz" Jul 06 23:56:41.963139 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:41.963110 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb4df\" (UniqueName: \"kubernetes.io/projected/8708739e-7b51-4394-9e97-89a8c1b87997-kube-api-access-rb4df\") pod \"kuadrant-operator-catalog-pmjrz\" (UID: \"8708739e-7b51-4394-9e97-89a8c1b87997\") " pod="kuadrant-system/kuadrant-operator-catalog-pmjrz" Jul 06 23:56:41.991849 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:41.991810 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pmjrz" Jul 06 23:56:42.142751 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:42.142726 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pmjrz"] Jul 06 23:56:42.161368 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:56:42.161334 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8708739e_7b51_4394_9e97_89a8c1b87997.slice/crio-8cca353879e3f60ddac9d0ac96686cb8f579872029941a4ce2f953becd36adf0 WatchSource:0}: Error finding container 8cca353879e3f60ddac9d0ac96686cb8f579872029941a4ce2f953becd36adf0: Status 404 returned error can't find the container with id 8cca353879e3f60ddac9d0ac96686cb8f579872029941a4ce2f953becd36adf0 Jul 06 23:56:42.592005 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:42.591971 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pmjrz" event={"ID":"8708739e-7b51-4394-9e97-89a8c1b87997","Type":"ContainerStarted","Data":"8cca353879e3f60ddac9d0ac96686cb8f579872029941a4ce2f953becd36adf0"} Jul 06 23:56:44.602739 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:44.602696 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r297p" event={"ID":"c20a4f0c-44fe-40ca-8461-57cca8b4c534","Type":"ContainerStarted","Data":"f6e37de5cb81c7df925c5aff37b660f35acd827ec2b5274d20b18392de40eec9"} Jul 06 23:56:44.603291 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:44.602756 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-r297p" podUID="c20a4f0c-44fe-40ca-8461-57cca8b4c534" containerName="registry-server" containerID="cri-o://f6e37de5cb81c7df925c5aff37b660f35acd827ec2b5274d20b18392de40eec9" gracePeriod=2 Jul 06 23:56:44.604307 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:44.604272 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pmjrz" event={"ID":"8708739e-7b51-4394-9e97-89a8c1b87997","Type":"ContainerStarted","Data":"ea517efcfb0f41921284ec04af49102d3dd8f3bcb7f4a0d60b6f3c5d104774ae"} Jul 06 23:56:44.619825 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:44.619773 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-r297p" podStartSLOduration=1.627747249 podStartE2EDuration="3.619752182s" totalImagesPullingTime="1.992004933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:44.618422626 +0000 UTC m=+486.828152167" watchObservedRunningTime="2026-07-06 23:56:44.619752182 +0000 UTC m=+486.829481702" Jul 06 23:56:44.636547 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:44.636495 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-pmjrz" podStartSLOduration=2.245172639 podStartE2EDuration="3.636476644s" totalImagesPullingTime="1.391304005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:44.635281715 +0000 UTC m=+486.845011255" watchObservedRunningTime="2026-07-06 23:56:44.636476644 +0000 UTC m=+486.846206164" Jul 06 23:56:44.854705 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:44.854643 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r297p" Jul 06 23:56:44.985636 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:44.985599 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7f9w\" (UniqueName: \"kubernetes.io/projected/c20a4f0c-44fe-40ca-8461-57cca8b4c534-kube-api-access-c7f9w\") pod \"c20a4f0c-44fe-40ca-8461-57cca8b4c534\" (UID: \"c20a4f0c-44fe-40ca-8461-57cca8b4c534\") " Jul 06 23:56:44.987765 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:44.987734 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c20a4f0c-44fe-40ca-8461-57cca8b4c534-kube-api-access-c7f9w" (OuterVolumeSpecName: "kube-api-access-c7f9w") pod "c20a4f0c-44fe-40ca-8461-57cca8b4c534" (UID: "c20a4f0c-44fe-40ca-8461-57cca8b4c534"). InnerVolumeSpecName "kube-api-access-c7f9w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:56:45.087154 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:45.087118 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7f9w\" (UniqueName: \"kubernetes.io/projected/c20a4f0c-44fe-40ca-8461-57cca8b4c534-kube-api-access-c7f9w\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:56:45.610631 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:45.610591 2553 generic.go:356] "Generic (PLEG): container finished" podID="c20a4f0c-44fe-40ca-8461-57cca8b4c534" containerID="f6e37de5cb81c7df925c5aff37b660f35acd827ec2b5274d20b18392de40eec9" exitCode=0 Jul 06 23:56:45.611035 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:45.610638 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r297p" Jul 06 23:56:45.611035 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:45.610676 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r297p" event={"ID":"c20a4f0c-44fe-40ca-8461-57cca8b4c534","Type":"ContainerDied","Data":"f6e37de5cb81c7df925c5aff37b660f35acd827ec2b5274d20b18392de40eec9"} Jul 06 23:56:45.611035 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:45.610734 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r297p" event={"ID":"c20a4f0c-44fe-40ca-8461-57cca8b4c534","Type":"ContainerDied","Data":"c4b0ddd9ac81893c8b0881c28392b78726333d57bdc2b039b4ee99f496084837"} Jul 06 23:56:45.611035 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:45.610755 2553 scope.go:117] "RemoveContainer" containerID="f6e37de5cb81c7df925c5aff37b660f35acd827ec2b5274d20b18392de40eec9" Jul 06 23:56:45.621632 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:45.621608 2553 scope.go:117] "RemoveContainer" containerID="f6e37de5cb81c7df925c5aff37b660f35acd827ec2b5274d20b18392de40eec9" Jul 06 23:56:45.621937 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:56:45.621917 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6e37de5cb81c7df925c5aff37b660f35acd827ec2b5274d20b18392de40eec9\": container with ID starting with f6e37de5cb81c7df925c5aff37b660f35acd827ec2b5274d20b18392de40eec9 not found: ID does not exist" containerID="f6e37de5cb81c7df925c5aff37b660f35acd827ec2b5274d20b18392de40eec9" Jul 06 23:56:45.621992 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:45.621946 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6e37de5cb81c7df925c5aff37b660f35acd827ec2b5274d20b18392de40eec9"} err="failed to get container status \"f6e37de5cb81c7df925c5aff37b660f35acd827ec2b5274d20b18392de40eec9\": rpc error: code = NotFound desc = could not find container \"f6e37de5cb81c7df925c5aff37b660f35acd827ec2b5274d20b18392de40eec9\": container with ID starting with f6e37de5cb81c7df925c5aff37b660f35acd827ec2b5274d20b18392de40eec9 not found: ID does not exist" Jul 06 23:56:45.633391 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:45.633362 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r297p"] Jul 06 23:56:45.635176 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:45.635156 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r297p"] Jul 06 23:56:46.301967 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:46.301936 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c20a4f0c-44fe-40ca-8461-57cca8b4c534" path="/var/lib/kubelet/pods/c20a4f0c-44fe-40ca-8461-57cca8b4c534/volumes" Jul 06 23:56:51.992453 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:51.992410 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-pmjrz" Jul 06 23:56:51.992886 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:51.992478 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-pmjrz" Jul 06 23:56:52.015441 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:52.015412 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-pmjrz" Jul 06 23:56:52.666702 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:52.666674 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-pmjrz" Jul 06 23:56:56.490866 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:56.490827 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s"] Jul 06 23:56:56.491260 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:56.491165 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c20a4f0c-44fe-40ca-8461-57cca8b4c534" containerName="registry-server" Jul 06 23:56:56.491260 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:56.491177 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="c20a4f0c-44fe-40ca-8461-57cca8b4c534" containerName="registry-server" Jul 06 23:56:56.491260 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:56.491252 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="c20a4f0c-44fe-40ca-8461-57cca8b4c534" containerName="registry-server" Jul 06 23:56:56.495865 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:56.495842 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s" Jul 06 23:56:56.498804 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:56.498782 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-fhz99\"" Jul 06 23:56:56.502574 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:56.502545 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s"] Jul 06 23:56:56.579232 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:56.579195 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b1c8289-6786-41c6-a0fd-66e613080735-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s\" (UID: \"7b1c8289-6786-41c6-a0fd-66e613080735\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s" Jul 06 23:56:56.579232 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:56.579234 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf6jt\" (UniqueName: \"kubernetes.io/projected/7b1c8289-6786-41c6-a0fd-66e613080735-kube-api-access-xf6jt\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s\" (UID: \"7b1c8289-6786-41c6-a0fd-66e613080735\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s" Jul 06 23:56:56.579506 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:56.579270 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b1c8289-6786-41c6-a0fd-66e613080735-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s\" (UID: \"7b1c8289-6786-41c6-a0fd-66e613080735\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s" Jul 06 23:56:56.680104 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:56.680065 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b1c8289-6786-41c6-a0fd-66e613080735-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s\" (UID: \"7b1c8289-6786-41c6-a0fd-66e613080735\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s" Jul 06 23:56:56.680104 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:56.680117 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf6jt\" (UniqueName: \"kubernetes.io/projected/7b1c8289-6786-41c6-a0fd-66e613080735-kube-api-access-xf6jt\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s\" (UID: \"7b1c8289-6786-41c6-a0fd-66e613080735\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s" Jul 06 23:56:56.680433 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:56.680153 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b1c8289-6786-41c6-a0fd-66e613080735-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s\" (UID: \"7b1c8289-6786-41c6-a0fd-66e613080735\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s" Jul 06 23:56:56.680567 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:56.680543 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b1c8289-6786-41c6-a0fd-66e613080735-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s\" (UID: \"7b1c8289-6786-41c6-a0fd-66e613080735\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s" Jul 06 23:56:56.680633 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:56.680551 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b1c8289-6786-41c6-a0fd-66e613080735-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s\" (UID: \"7b1c8289-6786-41c6-a0fd-66e613080735\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s" Jul 06 23:56:56.688120 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:56.688091 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf6jt\" (UniqueName: \"kubernetes.io/projected/7b1c8289-6786-41c6-a0fd-66e613080735-kube-api-access-xf6jt\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s\" (UID: \"7b1c8289-6786-41c6-a0fd-66e613080735\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s" Jul 06 23:56:56.806493 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:56.806407 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s" Jul 06 23:56:56.942420 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:56.942359 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s"] Jul 06 23:56:56.945176 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:56:56.945144 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b1c8289_6786_41c6_a0fd_66e613080735.slice/crio-fc7969d79d7455ba002c82fadb762b64cd1e2c2beb66d2ecf99e0306bb5bd3f2 WatchSource:0}: Error finding container fc7969d79d7455ba002c82fadb762b64cd1e2c2beb66d2ecf99e0306bb5bd3f2: Status 404 returned error can't find the container with id fc7969d79d7455ba002c82fadb762b64cd1e2c2beb66d2ecf99e0306bb5bd3f2 Jul 06 23:56:57.081383 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.081336 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v"] Jul 06 23:56:57.084980 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.084950 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v" Jul 06 23:56:57.094102 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.094073 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v"] Jul 06 23:56:57.185943 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.185905 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3ced335-f964-46a9-b311-969a96fc1136-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v\" (UID: \"a3ced335-f964-46a9-b311-969a96fc1136\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v" Jul 06 23:56:57.186101 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.185967 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3ced335-f964-46a9-b311-969a96fc1136-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v\" (UID: \"a3ced335-f964-46a9-b311-969a96fc1136\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v" Jul 06 23:56:57.186101 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.186037 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jbrk\" (UniqueName: \"kubernetes.io/projected/a3ced335-f964-46a9-b311-969a96fc1136-kube-api-access-5jbrk\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v\" (UID: \"a3ced335-f964-46a9-b311-969a96fc1136\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v" Jul 06 23:56:57.287496 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.287459 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3ced335-f964-46a9-b311-969a96fc1136-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v\" (UID: \"a3ced335-f964-46a9-b311-969a96fc1136\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v" Jul 06 23:56:57.287671 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.287515 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jbrk\" (UniqueName: \"kubernetes.io/projected/a3ced335-f964-46a9-b311-969a96fc1136-kube-api-access-5jbrk\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v\" (UID: \"a3ced335-f964-46a9-b311-969a96fc1136\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v" Jul 06 23:56:57.287671 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.287572 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3ced335-f964-46a9-b311-969a96fc1136-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v\" (UID: \"a3ced335-f964-46a9-b311-969a96fc1136\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v" Jul 06 23:56:57.287865 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.287847 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3ced335-f964-46a9-b311-969a96fc1136-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v\" (UID: \"a3ced335-f964-46a9-b311-969a96fc1136\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v" Jul 06 23:56:57.287902 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.287891 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3ced335-f964-46a9-b311-969a96fc1136-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v\" (UID: \"a3ced335-f964-46a9-b311-969a96fc1136\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v" Jul 06 23:56:57.295810 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.295778 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jbrk\" (UniqueName: \"kubernetes.io/projected/a3ced335-f964-46a9-b311-969a96fc1136-kube-api-access-5jbrk\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v\" (UID: \"a3ced335-f964-46a9-b311-969a96fc1136\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v" Jul 06 23:56:57.436903 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.436868 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v" Jul 06 23:56:57.497274 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.496409 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm"] Jul 06 23:56:57.510448 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.509546 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm"] Jul 06 23:56:57.510448 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.509692 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm" Jul 06 23:56:57.580932 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.580901 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v"] Jul 06 23:56:57.582973 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:56:57.582947 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3ced335_f964_46a9_b311_969a96fc1136.slice/crio-26f58a44ad6723f515805e52cd6b48a74e85d8450706903d6b44f679779853f9 WatchSource:0}: Error finding container 26f58a44ad6723f515805e52cd6b48a74e85d8450706903d6b44f679779853f9: Status 404 returned error can't find the container with id 26f58a44ad6723f515805e52cd6b48a74e85d8450706903d6b44f679779853f9 Jul 06 23:56:57.591076 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.591046 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4be0703-a52e-49d4-8942-edbb447df0ad-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm\" (UID: \"c4be0703-a52e-49d4-8942-edbb447df0ad\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm" Jul 06 23:56:57.591217 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.591086 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4be0703-a52e-49d4-8942-edbb447df0ad-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm\" (UID: \"c4be0703-a52e-49d4-8942-edbb447df0ad\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm" Jul 06 23:56:57.591217 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.591119 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg7jh\" (UniqueName: \"kubernetes.io/projected/c4be0703-a52e-49d4-8942-edbb447df0ad-kube-api-access-zg7jh\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm\" (UID: \"c4be0703-a52e-49d4-8942-edbb447df0ad\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm" Jul 06 23:56:57.676047 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.676014 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v" event={"ID":"a3ced335-f964-46a9-b311-969a96fc1136","Type":"ContainerStarted","Data":"e1bdfa2b6635045a334c93064d7ef518ae56d1203d0e70174aa9b70b9f583e7b"} Jul 06 23:56:57.676206 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.676076 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v" event={"ID":"a3ced335-f964-46a9-b311-969a96fc1136","Type":"ContainerStarted","Data":"26f58a44ad6723f515805e52cd6b48a74e85d8450706903d6b44f679779853f9"} Jul 06 23:56:57.677943 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.677899 2553 generic.go:356] "Generic (PLEG): container finished" podID="7b1c8289-6786-41c6-a0fd-66e613080735" containerID="8d028bc0536c0743ff21a1fbee5c03ee3b37ae3586d7fb9ce4f7eb10ddced5c8" exitCode=0 Jul 06 23:56:57.678056 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.677975 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s" event={"ID":"7b1c8289-6786-41c6-a0fd-66e613080735","Type":"ContainerDied","Data":"8d028bc0536c0743ff21a1fbee5c03ee3b37ae3586d7fb9ce4f7eb10ddced5c8"} Jul 06 23:56:57.678056 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.678022 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s" event={"ID":"7b1c8289-6786-41c6-a0fd-66e613080735","Type":"ContainerStarted","Data":"fc7969d79d7455ba002c82fadb762b64cd1e2c2beb66d2ecf99e0306bb5bd3f2"} Jul 06 23:56:57.691671 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.691599 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg7jh\" (UniqueName: \"kubernetes.io/projected/c4be0703-a52e-49d4-8942-edbb447df0ad-kube-api-access-zg7jh\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm\" (UID: \"c4be0703-a52e-49d4-8942-edbb447df0ad\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm" Jul 06 23:56:57.691823 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.691712 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4be0703-a52e-49d4-8942-edbb447df0ad-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm\" (UID: \"c4be0703-a52e-49d4-8942-edbb447df0ad\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm" Jul 06 23:56:57.691823 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.691752 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4be0703-a52e-49d4-8942-edbb447df0ad-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm\" (UID: \"c4be0703-a52e-49d4-8942-edbb447df0ad\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm" Jul 06 23:56:57.692059 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.692035 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4be0703-a52e-49d4-8942-edbb447df0ad-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm\" (UID: \"c4be0703-a52e-49d4-8942-edbb447df0ad\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm" Jul 06 23:56:57.692142 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.692071 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4be0703-a52e-49d4-8942-edbb447df0ad-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm\" (UID: \"c4be0703-a52e-49d4-8942-edbb447df0ad\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm" Jul 06 23:56:57.700615 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.700587 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg7jh\" (UniqueName: \"kubernetes.io/projected/c4be0703-a52e-49d4-8942-edbb447df0ad-kube-api-access-zg7jh\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm\" (UID: \"c4be0703-a52e-49d4-8942-edbb447df0ad\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm" Jul 06 23:56:57.819870 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.819831 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm" Jul 06 23:56:57.886480 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.886447 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c"] Jul 06 23:56:57.891353 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.891323 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c" Jul 06 23:56:57.898643 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.898610 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c"] Jul 06 23:56:57.967306 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.967265 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm"] Jul 06 23:56:57.969327 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:56:57.969278 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4be0703_a52e_49d4_8942_edbb447df0ad.slice/crio-74190d49ca6304dffec695a4107970af1aa9ef53ca38f534a0a43ef54e8d043a WatchSource:0}: Error finding container 74190d49ca6304dffec695a4107970af1aa9ef53ca38f534a0a43ef54e8d043a: Status 404 returned error can't find the container with id 74190d49ca6304dffec695a4107970af1aa9ef53ca38f534a0a43ef54e8d043a Jul 06 23:56:57.994711 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.994682 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86qq8\" (UniqueName: \"kubernetes.io/projected/d4edb7a7-2478-43e1-b346-d786e1606487-kube-api-access-86qq8\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c\" (UID: \"d4edb7a7-2478-43e1-b346-d786e1606487\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c" Jul 06 23:56:57.994847 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.994826 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4edb7a7-2478-43e1-b346-d786e1606487-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c\" (UID: \"d4edb7a7-2478-43e1-b346-d786e1606487\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c" Jul 06 23:56:57.994919 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:57.994860 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4edb7a7-2478-43e1-b346-d786e1606487-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c\" (UID: \"d4edb7a7-2478-43e1-b346-d786e1606487\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c" Jul 06 23:56:58.096174 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:58.096144 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4edb7a7-2478-43e1-b346-d786e1606487-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c\" (UID: \"d4edb7a7-2478-43e1-b346-d786e1606487\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c" Jul 06 23:56:58.096320 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:58.096195 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4edb7a7-2478-43e1-b346-d786e1606487-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c\" (UID: \"d4edb7a7-2478-43e1-b346-d786e1606487\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c" Jul 06 23:56:58.096320 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:58.096251 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86qq8\" (UniqueName: \"kubernetes.io/projected/d4edb7a7-2478-43e1-b346-d786e1606487-kube-api-access-86qq8\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c\" (UID: \"d4edb7a7-2478-43e1-b346-d786e1606487\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c" Jul 06 23:56:58.096615 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:58.096593 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4edb7a7-2478-43e1-b346-d786e1606487-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c\" (UID: \"d4edb7a7-2478-43e1-b346-d786e1606487\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c" Jul 06 23:56:58.096615 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:58.096625 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4edb7a7-2478-43e1-b346-d786e1606487-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c\" (UID: \"d4edb7a7-2478-43e1-b346-d786e1606487\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c" Jul 06 23:56:58.106016 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:58.105990 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86qq8\" (UniqueName: \"kubernetes.io/projected/d4edb7a7-2478-43e1-b346-d786e1606487-kube-api-access-86qq8\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c\" (UID: \"d4edb7a7-2478-43e1-b346-d786e1606487\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c" Jul 06 23:56:58.203450 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:58.203410 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c" Jul 06 23:56:58.372150 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:58.372102 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c"] Jul 06 23:56:58.374729 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:56:58.374694 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4edb7a7_2478_43e1_b346_d786e1606487.slice/crio-97e856eb402b145bd662fe1bbadfd0f54aee89e4be4aeea2ba9435230fc670d8 WatchSource:0}: Error finding container 97e856eb402b145bd662fe1bbadfd0f54aee89e4be4aeea2ba9435230fc670d8: Status 404 returned error can't find the container with id 97e856eb402b145bd662fe1bbadfd0f54aee89e4be4aeea2ba9435230fc670d8 Jul 06 23:56:58.683947 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:58.683910 2553 generic.go:356] "Generic (PLEG): container finished" podID="d4edb7a7-2478-43e1-b346-d786e1606487" containerID="40160d73b8bd70b2ca413c2613f9f2fd6fe0f8f918996222bd19f25787e997f8" exitCode=0 Jul 06 23:56:58.684462 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:58.683996 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c" event={"ID":"d4edb7a7-2478-43e1-b346-d786e1606487","Type":"ContainerDied","Data":"40160d73b8bd70b2ca413c2613f9f2fd6fe0f8f918996222bd19f25787e997f8"} Jul 06 23:56:58.684462 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:58.684051 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c" event={"ID":"d4edb7a7-2478-43e1-b346-d786e1606487","Type":"ContainerStarted","Data":"97e856eb402b145bd662fe1bbadfd0f54aee89e4be4aeea2ba9435230fc670d8"} Jul 06 23:56:58.685725 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:58.685703 2553 generic.go:356] "Generic (PLEG): container finished" podID="a3ced335-f964-46a9-b311-969a96fc1136" containerID="e1bdfa2b6635045a334c93064d7ef518ae56d1203d0e70174aa9b70b9f583e7b" exitCode=0 Jul 06 23:56:58.685900 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:58.685786 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v" event={"ID":"a3ced335-f964-46a9-b311-969a96fc1136","Type":"ContainerDied","Data":"e1bdfa2b6635045a334c93064d7ef518ae56d1203d0e70174aa9b70b9f583e7b"} Jul 06 23:56:58.687405 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:58.687383 2553 generic.go:356] "Generic (PLEG): container finished" podID="c4be0703-a52e-49d4-8942-edbb447df0ad" containerID="bf3b80605baec8306eab59e093150772885cd6ee3e2a5082e97bd339f65c3549" exitCode=0 Jul 06 23:56:58.687511 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:58.687475 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm" event={"ID":"c4be0703-a52e-49d4-8942-edbb447df0ad","Type":"ContainerDied","Data":"bf3b80605baec8306eab59e093150772885cd6ee3e2a5082e97bd339f65c3549"} Jul 06 23:56:58.687564 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:58.687520 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm" event={"ID":"c4be0703-a52e-49d4-8942-edbb447df0ad","Type":"ContainerStarted","Data":"74190d49ca6304dffec695a4107970af1aa9ef53ca38f534a0a43ef54e8d043a"} Jul 06 23:56:58.689670 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:58.689650 2553 generic.go:356] "Generic (PLEG): container finished" podID="7b1c8289-6786-41c6-a0fd-66e613080735" containerID="976f986c523b5f2c9099a1f0fff95911cb0f0127a4daf7fcc7b49b09657a5d37" exitCode=0 Jul 06 23:56:58.689746 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:58.689705 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s" event={"ID":"7b1c8289-6786-41c6-a0fd-66e613080735","Type":"ContainerDied","Data":"976f986c523b5f2c9099a1f0fff95911cb0f0127a4daf7fcc7b49b09657a5d37"} Jul 06 23:56:59.701719 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:59.701666 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm" event={"ID":"c4be0703-a52e-49d4-8942-edbb447df0ad","Type":"ContainerStarted","Data":"6264000cec5b0f7cf660daad04efbc07b26fd734dbd4f4dd7d7434463cbbd2ff"} Jul 06 23:56:59.705118 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:59.705086 2553 generic.go:356] "Generic (PLEG): container finished" podID="7b1c8289-6786-41c6-a0fd-66e613080735" containerID="b3a62a6ba033ae25137d65eb8d4a0a6a52900079a7d7ac8787b850de42259a27" exitCode=0 Jul 06 23:56:59.705285 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:56:59.705165 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s" event={"ID":"7b1c8289-6786-41c6-a0fd-66e613080735","Type":"ContainerDied","Data":"b3a62a6ba033ae25137d65eb8d4a0a6a52900079a7d7ac8787b850de42259a27"} Jul 06 23:57:00.228324 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.228275 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76d6c4db4b-qvwbm"] Jul 06 23:57:00.231994 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.231972 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76d6c4db4b-qvwbm" Jul 06 23:57:00.244615 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.244580 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76d6c4db4b-qvwbm"] Jul 06 23:57:00.318122 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.318086 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwz9m\" (UniqueName: \"kubernetes.io/projected/61611271-9dd7-4ef2-ab0c-9367e8cee39e-kube-api-access-pwz9m\") pod \"console-76d6c4db4b-qvwbm\" (UID: \"61611271-9dd7-4ef2-ab0c-9367e8cee39e\") " pod="openshift-console/console-76d6c4db4b-qvwbm" Jul 06 23:57:00.318340 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.318127 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/61611271-9dd7-4ef2-ab0c-9367e8cee39e-console-serving-cert\") pod \"console-76d6c4db4b-qvwbm\" (UID: \"61611271-9dd7-4ef2-ab0c-9367e8cee39e\") " pod="openshift-console/console-76d6c4db4b-qvwbm" Jul 06 23:57:00.318340 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.318169 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/61611271-9dd7-4ef2-ab0c-9367e8cee39e-console-config\") pod \"console-76d6c4db4b-qvwbm\" (UID: \"61611271-9dd7-4ef2-ab0c-9367e8cee39e\") " pod="openshift-console/console-76d6c4db4b-qvwbm" Jul 06 23:57:00.318340 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.318275 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/61611271-9dd7-4ef2-ab0c-9367e8cee39e-console-oauth-config\") pod \"console-76d6c4db4b-qvwbm\" (UID: \"61611271-9dd7-4ef2-ab0c-9367e8cee39e\") " pod="openshift-console/console-76d6c4db4b-qvwbm" Jul 06 23:57:00.318455 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.318339 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61611271-9dd7-4ef2-ab0c-9367e8cee39e-trusted-ca-bundle\") pod \"console-76d6c4db4b-qvwbm\" (UID: \"61611271-9dd7-4ef2-ab0c-9367e8cee39e\") " pod="openshift-console/console-76d6c4db4b-qvwbm" Jul 06 23:57:00.318455 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.318364 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/61611271-9dd7-4ef2-ab0c-9367e8cee39e-service-ca\") pod \"console-76d6c4db4b-qvwbm\" (UID: \"61611271-9dd7-4ef2-ab0c-9367e8cee39e\") " pod="openshift-console/console-76d6c4db4b-qvwbm" Jul 06 23:57:00.318455 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.318382 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/61611271-9dd7-4ef2-ab0c-9367e8cee39e-oauth-serving-cert\") pod \"console-76d6c4db4b-qvwbm\" (UID: \"61611271-9dd7-4ef2-ab0c-9367e8cee39e\") " pod="openshift-console/console-76d6c4db4b-qvwbm" Jul 06 23:57:00.419205 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.419168 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/61611271-9dd7-4ef2-ab0c-9367e8cee39e-service-ca\") pod \"console-76d6c4db4b-qvwbm\" (UID: \"61611271-9dd7-4ef2-ab0c-9367e8cee39e\") " pod="openshift-console/console-76d6c4db4b-qvwbm" Jul 06 23:57:00.419205 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.419209 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/61611271-9dd7-4ef2-ab0c-9367e8cee39e-oauth-serving-cert\") pod \"console-76d6c4db4b-qvwbm\" (UID: \"61611271-9dd7-4ef2-ab0c-9367e8cee39e\") " pod="openshift-console/console-76d6c4db4b-qvwbm" Jul 06 23:57:00.419440 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.419239 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwz9m\" (UniqueName: \"kubernetes.io/projected/61611271-9dd7-4ef2-ab0c-9367e8cee39e-kube-api-access-pwz9m\") pod \"console-76d6c4db4b-qvwbm\" (UID: \"61611271-9dd7-4ef2-ab0c-9367e8cee39e\") " pod="openshift-console/console-76d6c4db4b-qvwbm" Jul 06 23:57:00.419440 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.419363 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/61611271-9dd7-4ef2-ab0c-9367e8cee39e-console-serving-cert\") pod \"console-76d6c4db4b-qvwbm\" (UID: \"61611271-9dd7-4ef2-ab0c-9367e8cee39e\") " pod="openshift-console/console-76d6c4db4b-qvwbm" Jul 06 23:57:00.419440 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.419429 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/61611271-9dd7-4ef2-ab0c-9367e8cee39e-console-config\") pod \"console-76d6c4db4b-qvwbm\" (UID: \"61611271-9dd7-4ef2-ab0c-9367e8cee39e\") " pod="openshift-console/console-76d6c4db4b-qvwbm" Jul 06 23:57:00.419576 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.419539 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/61611271-9dd7-4ef2-ab0c-9367e8cee39e-console-oauth-config\") pod \"console-76d6c4db4b-qvwbm\" (UID: \"61611271-9dd7-4ef2-ab0c-9367e8cee39e\") " pod="openshift-console/console-76d6c4db4b-qvwbm" Jul 06 23:57:00.419576 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.419571 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61611271-9dd7-4ef2-ab0c-9367e8cee39e-trusted-ca-bundle\") pod \"console-76d6c4db4b-qvwbm\" (UID: \"61611271-9dd7-4ef2-ab0c-9367e8cee39e\") " pod="openshift-console/console-76d6c4db4b-qvwbm" Jul 06 23:57:00.420025 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.419997 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/61611271-9dd7-4ef2-ab0c-9367e8cee39e-service-ca\") pod \"console-76d6c4db4b-qvwbm\" (UID: \"61611271-9dd7-4ef2-ab0c-9367e8cee39e\") " pod="openshift-console/console-76d6c4db4b-qvwbm" Jul 06 23:57:00.420161 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.420089 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/61611271-9dd7-4ef2-ab0c-9367e8cee39e-oauth-serving-cert\") pod \"console-76d6c4db4b-qvwbm\" (UID: \"61611271-9dd7-4ef2-ab0c-9367e8cee39e\") " pod="openshift-console/console-76d6c4db4b-qvwbm" Jul 06 23:57:00.420161 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.420097 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/61611271-9dd7-4ef2-ab0c-9367e8cee39e-console-config\") pod \"console-76d6c4db4b-qvwbm\" (UID: \"61611271-9dd7-4ef2-ab0c-9367e8cee39e\") " pod="openshift-console/console-76d6c4db4b-qvwbm" Jul 06 23:57:00.420374 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.420355 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61611271-9dd7-4ef2-ab0c-9367e8cee39e-trusted-ca-bundle\") pod \"console-76d6c4db4b-qvwbm\" (UID: \"61611271-9dd7-4ef2-ab0c-9367e8cee39e\") " pod="openshift-console/console-76d6c4db4b-qvwbm" Jul 06 23:57:00.421954 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.421925 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/61611271-9dd7-4ef2-ab0c-9367e8cee39e-console-oauth-config\") pod \"console-76d6c4db4b-qvwbm\" (UID: \"61611271-9dd7-4ef2-ab0c-9367e8cee39e\") " pod="openshift-console/console-76d6c4db4b-qvwbm" Jul 06 23:57:00.421954 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.421952 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/61611271-9dd7-4ef2-ab0c-9367e8cee39e-console-serving-cert\") pod \"console-76d6c4db4b-qvwbm\" (UID: \"61611271-9dd7-4ef2-ab0c-9367e8cee39e\") " pod="openshift-console/console-76d6c4db4b-qvwbm" Jul 06 23:57:00.427569 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.427549 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwz9m\" (UniqueName: \"kubernetes.io/projected/61611271-9dd7-4ef2-ab0c-9367e8cee39e-kube-api-access-pwz9m\") pod \"console-76d6c4db4b-qvwbm\" (UID: \"61611271-9dd7-4ef2-ab0c-9367e8cee39e\") " pod="openshift-console/console-76d6c4db4b-qvwbm" Jul 06 23:57:00.541786 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.541679 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76d6c4db4b-qvwbm" Jul 06 23:57:00.685572 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.685514 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76d6c4db4b-qvwbm"] Jul 06 23:57:00.688412 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:57:00.688386 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61611271_9dd7_4ef2_ab0c_9367e8cee39e.slice/crio-91fa6242ea68848b5bff958784b35f94b00233d911dbb8ef20d7a94ce4b0565a WatchSource:0}: Error finding container 91fa6242ea68848b5bff958784b35f94b00233d911dbb8ef20d7a94ce4b0565a: Status 404 returned error can't find the container with id 91fa6242ea68848b5bff958784b35f94b00233d911dbb8ef20d7a94ce4b0565a Jul 06 23:57:00.714498 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.714466 2553 generic.go:356] "Generic (PLEG): container finished" podID="d4edb7a7-2478-43e1-b346-d786e1606487" containerID="c68bb4df9bb1b4772ebbf54dfebe09620236e0acefc37a3b16f6a5634fada293" exitCode=0 Jul 06 23:57:00.714935 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.714540 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c" event={"ID":"d4edb7a7-2478-43e1-b346-d786e1606487","Type":"ContainerDied","Data":"c68bb4df9bb1b4772ebbf54dfebe09620236e0acefc37a3b16f6a5634fada293"} Jul 06 23:57:00.716738 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.716711 2553 generic.go:356] "Generic (PLEG): container finished" podID="a3ced335-f964-46a9-b311-969a96fc1136" containerID="ee6eaafd2daba8483218479e3b70fd52884e20c01bc2917b3a7110cc3f170902" exitCode=0 Jul 06 23:57:00.716872 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.716740 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v" event={"ID":"a3ced335-f964-46a9-b311-969a96fc1136","Type":"ContainerDied","Data":"ee6eaafd2daba8483218479e3b70fd52884e20c01bc2917b3a7110cc3f170902"} Jul 06 23:57:00.718910 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.718884 2553 generic.go:356] "Generic (PLEG): container finished" podID="c4be0703-a52e-49d4-8942-edbb447df0ad" containerID="6264000cec5b0f7cf660daad04efbc07b26fd734dbd4f4dd7d7434463cbbd2ff" exitCode=0 Jul 06 23:57:00.719024 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.718964 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm" event={"ID":"c4be0703-a52e-49d4-8942-edbb447df0ad","Type":"ContainerDied","Data":"6264000cec5b0f7cf660daad04efbc07b26fd734dbd4f4dd7d7434463cbbd2ff"} Jul 06 23:57:00.720315 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:00.720270 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76d6c4db4b-qvwbm" event={"ID":"61611271-9dd7-4ef2-ab0c-9367e8cee39e","Type":"ContainerStarted","Data":"91fa6242ea68848b5bff958784b35f94b00233d911dbb8ef20d7a94ce4b0565a"} Jul 06 23:57:01.038431 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:01.038395 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s" Jul 06 23:57:01.126731 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:01.126637 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b1c8289-6786-41c6-a0fd-66e613080735-util\") pod \"7b1c8289-6786-41c6-a0fd-66e613080735\" (UID: \"7b1c8289-6786-41c6-a0fd-66e613080735\") " Jul 06 23:57:01.126731 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:01.126681 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b1c8289-6786-41c6-a0fd-66e613080735-bundle\") pod \"7b1c8289-6786-41c6-a0fd-66e613080735\" (UID: \"7b1c8289-6786-41c6-a0fd-66e613080735\") " Jul 06 23:57:01.126731 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:01.126706 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xf6jt\" (UniqueName: \"kubernetes.io/projected/7b1c8289-6786-41c6-a0fd-66e613080735-kube-api-access-xf6jt\") pod \"7b1c8289-6786-41c6-a0fd-66e613080735\" (UID: \"7b1c8289-6786-41c6-a0fd-66e613080735\") " Jul 06 23:57:01.127274 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:01.127247 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b1c8289-6786-41c6-a0fd-66e613080735-bundle" (OuterVolumeSpecName: "bundle") pod "7b1c8289-6786-41c6-a0fd-66e613080735" (UID: "7b1c8289-6786-41c6-a0fd-66e613080735"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:57:01.128869 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:01.128841 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b1c8289-6786-41c6-a0fd-66e613080735-kube-api-access-xf6jt" (OuterVolumeSpecName: "kube-api-access-xf6jt") pod "7b1c8289-6786-41c6-a0fd-66e613080735" (UID: "7b1c8289-6786-41c6-a0fd-66e613080735"). InnerVolumeSpecName "kube-api-access-xf6jt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:57:01.134325 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:01.134282 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b1c8289-6786-41c6-a0fd-66e613080735-util" (OuterVolumeSpecName: "util") pod "7b1c8289-6786-41c6-a0fd-66e613080735" (UID: "7b1c8289-6786-41c6-a0fd-66e613080735"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:57:01.227688 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:01.227641 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xf6jt\" (UniqueName: \"kubernetes.io/projected/7b1c8289-6786-41c6-a0fd-66e613080735-kube-api-access-xf6jt\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:57:01.227688 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:01.227680 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b1c8289-6786-41c6-a0fd-66e613080735-util\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:57:01.227688 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:01.227693 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b1c8289-6786-41c6-a0fd-66e613080735-bundle\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:57:01.726821 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:01.726774 2553 generic.go:356] "Generic (PLEG): container finished" podID="a3ced335-f964-46a9-b311-969a96fc1136" containerID="27a5703d9c6cff7493736c36cc896c287e9574c9d4967f2dbfb938e2605bd429" exitCode=0 Jul 06 23:57:01.727326 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:01.726830 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v" event={"ID":"a3ced335-f964-46a9-b311-969a96fc1136","Type":"ContainerDied","Data":"27a5703d9c6cff7493736c36cc896c287e9574c9d4967f2dbfb938e2605bd429"} Jul 06 23:57:01.728899 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:01.728874 2553 generic.go:356] "Generic (PLEG): container finished" podID="c4be0703-a52e-49d4-8942-edbb447df0ad" containerID="1650dee3724becac3e5e31bb22a176cf8b2566b776448cefa82be49ff54951e5" exitCode=0 Jul 06 23:57:01.729008 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:01.728936 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm" event={"ID":"c4be0703-a52e-49d4-8942-edbb447df0ad","Type":"ContainerDied","Data":"1650dee3724becac3e5e31bb22a176cf8b2566b776448cefa82be49ff54951e5"} Jul 06 23:57:01.730352 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:01.730328 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76d6c4db4b-qvwbm" event={"ID":"61611271-9dd7-4ef2-ab0c-9367e8cee39e","Type":"ContainerStarted","Data":"a1ba08ef8ebdd367553a9de47523e7201cbdd0875495d98a00bf93d802839d16"} Jul 06 23:57:01.732097 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:01.732077 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s" Jul 06 23:57:01.732097 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:01.732087 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s" event={"ID":"7b1c8289-6786-41c6-a0fd-66e613080735","Type":"ContainerDied","Data":"fc7969d79d7455ba002c82fadb762b64cd1e2c2beb66d2ecf99e0306bb5bd3f2"} Jul 06 23:57:01.732290 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:01.732130 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc7969d79d7455ba002c82fadb762b64cd1e2c2beb66d2ecf99e0306bb5bd3f2" Jul 06 23:57:01.734154 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:01.734134 2553 generic.go:356] "Generic (PLEG): container finished" podID="d4edb7a7-2478-43e1-b346-d786e1606487" containerID="126841d9d3c9e8ce79733abdc014becf0489a1ec34af41dfcee3216a2a7b8cd5" exitCode=0 Jul 06 23:57:01.734243 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:01.734182 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c" event={"ID":"d4edb7a7-2478-43e1-b346-d786e1606487","Type":"ContainerDied","Data":"126841d9d3c9e8ce79733abdc014becf0489a1ec34af41dfcee3216a2a7b8cd5"} Jul 06 23:57:01.783873 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:01.783815 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76d6c4db4b-qvwbm" podStartSLOduration=1.7837978639999998 podStartE2EDuration="1.783797864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:01.782067411 +0000 UTC m=+503.991796923" watchObservedRunningTime="2026-07-06 23:57:01.783797864 +0000 UTC m=+503.993527385" Jul 06 23:57:02.887593 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:02.887565 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm" Jul 06 23:57:02.935286 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:02.935245 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c" Jul 06 23:57:02.939854 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:02.939808 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v" Jul 06 23:57:02.941772 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:02.941753 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4be0703-a52e-49d4-8942-edbb447df0ad-bundle\") pod \"c4be0703-a52e-49d4-8942-edbb447df0ad\" (UID: \"c4be0703-a52e-49d4-8942-edbb447df0ad\") " Jul 06 23:57:02.941849 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:02.941788 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4be0703-a52e-49d4-8942-edbb447df0ad-util\") pod \"c4be0703-a52e-49d4-8942-edbb447df0ad\" (UID: \"c4be0703-a52e-49d4-8942-edbb447df0ad\") " Jul 06 23:57:02.941913 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:02.941870 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zg7jh\" (UniqueName: \"kubernetes.io/projected/c4be0703-a52e-49d4-8942-edbb447df0ad-kube-api-access-zg7jh\") pod \"c4be0703-a52e-49d4-8942-edbb447df0ad\" (UID: \"c4be0703-a52e-49d4-8942-edbb447df0ad\") " Jul 06 23:57:02.942256 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:02.942231 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4be0703-a52e-49d4-8942-edbb447df0ad-bundle" (OuterVolumeSpecName: "bundle") pod "c4be0703-a52e-49d4-8942-edbb447df0ad" (UID: "c4be0703-a52e-49d4-8942-edbb447df0ad"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:57:02.943978 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:02.943958 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4be0703-a52e-49d4-8942-edbb447df0ad-kube-api-access-zg7jh" (OuterVolumeSpecName: "kube-api-access-zg7jh") pod "c4be0703-a52e-49d4-8942-edbb447df0ad" (UID: "c4be0703-a52e-49d4-8942-edbb447df0ad"). InnerVolumeSpecName "kube-api-access-zg7jh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:57:02.947635 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:02.947607 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4be0703-a52e-49d4-8942-edbb447df0ad-util" (OuterVolumeSpecName: "util") pod "c4be0703-a52e-49d4-8942-edbb447df0ad" (UID: "c4be0703-a52e-49d4-8942-edbb447df0ad"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:57:03.043335 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.043203 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4edb7a7-2478-43e1-b346-d786e1606487-bundle\") pod \"d4edb7a7-2478-43e1-b346-d786e1606487\" (UID: \"d4edb7a7-2478-43e1-b346-d786e1606487\") " Jul 06 23:57:03.043335 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.043257 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86qq8\" (UniqueName: \"kubernetes.io/projected/d4edb7a7-2478-43e1-b346-d786e1606487-kube-api-access-86qq8\") pod \"d4edb7a7-2478-43e1-b346-d786e1606487\" (UID: \"d4edb7a7-2478-43e1-b346-d786e1606487\") " Jul 06 23:57:03.043335 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.043334 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4edb7a7-2478-43e1-b346-d786e1606487-util\") pod \"d4edb7a7-2478-43e1-b346-d786e1606487\" (UID: \"d4edb7a7-2478-43e1-b346-d786e1606487\") " Jul 06 23:57:03.043626 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.043361 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3ced335-f964-46a9-b311-969a96fc1136-bundle\") pod \"a3ced335-f964-46a9-b311-969a96fc1136\" (UID: \"a3ced335-f964-46a9-b311-969a96fc1136\") " Jul 06 23:57:03.043626 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.043406 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jbrk\" (UniqueName: \"kubernetes.io/projected/a3ced335-f964-46a9-b311-969a96fc1136-kube-api-access-5jbrk\") pod \"a3ced335-f964-46a9-b311-969a96fc1136\" (UID: \"a3ced335-f964-46a9-b311-969a96fc1136\") " Jul 06 23:57:03.043626 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.043436 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3ced335-f964-46a9-b311-969a96fc1136-util\") pod \"a3ced335-f964-46a9-b311-969a96fc1136\" (UID: \"a3ced335-f964-46a9-b311-969a96fc1136\") " Jul 06 23:57:03.043780 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.043659 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4be0703-a52e-49d4-8942-edbb447df0ad-bundle\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:57:03.043780 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.043676 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4be0703-a52e-49d4-8942-edbb447df0ad-util\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:57:03.043780 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.043690 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zg7jh\" (UniqueName: \"kubernetes.io/projected/c4be0703-a52e-49d4-8942-edbb447df0ad-kube-api-access-zg7jh\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:57:03.043927 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.043881 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4edb7a7-2478-43e1-b346-d786e1606487-bundle" (OuterVolumeSpecName: "bundle") pod "d4edb7a7-2478-43e1-b346-d786e1606487" (UID: "d4edb7a7-2478-43e1-b346-d786e1606487"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:57:03.044155 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.044130 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3ced335-f964-46a9-b311-969a96fc1136-bundle" (OuterVolumeSpecName: "bundle") pod "a3ced335-f964-46a9-b311-969a96fc1136" (UID: "a3ced335-f964-46a9-b311-969a96fc1136"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:57:03.045676 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.045650 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3ced335-f964-46a9-b311-969a96fc1136-kube-api-access-5jbrk" (OuterVolumeSpecName: "kube-api-access-5jbrk") pod "a3ced335-f964-46a9-b311-969a96fc1136" (UID: "a3ced335-f964-46a9-b311-969a96fc1136"). InnerVolumeSpecName "kube-api-access-5jbrk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:57:03.045816 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.045795 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4edb7a7-2478-43e1-b346-d786e1606487-kube-api-access-86qq8" (OuterVolumeSpecName: "kube-api-access-86qq8") pod "d4edb7a7-2478-43e1-b346-d786e1606487" (UID: "d4edb7a7-2478-43e1-b346-d786e1606487"). InnerVolumeSpecName "kube-api-access-86qq8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:57:03.048562 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.048541 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3ced335-f964-46a9-b311-969a96fc1136-util" (OuterVolumeSpecName: "util") pod "a3ced335-f964-46a9-b311-969a96fc1136" (UID: "a3ced335-f964-46a9-b311-969a96fc1136"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:57:03.049827 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.049804 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4edb7a7-2478-43e1-b346-d786e1606487-util" (OuterVolumeSpecName: "util") pod "d4edb7a7-2478-43e1-b346-d786e1606487" (UID: "d4edb7a7-2478-43e1-b346-d786e1606487"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:57:03.144187 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.144140 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d4edb7a7-2478-43e1-b346-d786e1606487-bundle\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:57:03.144187 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.144178 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86qq8\" (UniqueName: \"kubernetes.io/projected/d4edb7a7-2478-43e1-b346-d786e1606487-kube-api-access-86qq8\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:57:03.144187 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.144192 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d4edb7a7-2478-43e1-b346-d786e1606487-util\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:57:03.144482 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.144203 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3ced335-f964-46a9-b311-969a96fc1136-bundle\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:57:03.144482 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.144226 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jbrk\" (UniqueName: \"kubernetes.io/projected/a3ced335-f964-46a9-b311-969a96fc1136-kube-api-access-5jbrk\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:57:03.144482 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.144238 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3ced335-f964-46a9-b311-969a96fc1136-util\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:57:03.746937 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.746910 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v" Jul 06 23:57:03.746937 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.746915 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v" event={"ID":"a3ced335-f964-46a9-b311-969a96fc1136","Type":"ContainerDied","Data":"26f58a44ad6723f515805e52cd6b48a74e85d8450706903d6b44f679779853f9"} Jul 06 23:57:03.747163 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.746967 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="26f58a44ad6723f515805e52cd6b48a74e85d8450706903d6b44f679779853f9" Jul 06 23:57:03.748996 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.748967 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm" Jul 06 23:57:03.748996 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.748987 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm" event={"ID":"c4be0703-a52e-49d4-8942-edbb447df0ad","Type":"ContainerDied","Data":"74190d49ca6304dffec695a4107970af1aa9ef53ca38f534a0a43ef54e8d043a"} Jul 06 23:57:03.749214 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.749027 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="74190d49ca6304dffec695a4107970af1aa9ef53ca38f534a0a43ef54e8d043a" Jul 06 23:57:03.750879 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.750859 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c" event={"ID":"d4edb7a7-2478-43e1-b346-d786e1606487","Type":"ContainerDied","Data":"97e856eb402b145bd662fe1bbadfd0f54aee89e4be4aeea2ba9435230fc670d8"} Jul 06 23:57:03.750982 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.750890 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c" Jul 06 23:57:03.750982 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:03.750892 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="97e856eb402b145bd662fe1bbadfd0f54aee89e4be4aeea2ba9435230fc670d8" Jul 06 23:57:10.542792 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:10.542756 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76d6c4db4b-qvwbm" Jul 06 23:57:10.543231 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:10.542850 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76d6c4db4b-qvwbm" Jul 06 23:57:10.547325 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:10.547285 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-76d6c4db4b-qvwbm" Jul 06 23:57:10.795152 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:10.795076 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-76d6c4db4b-qvwbm" Jul 06 23:57:10.848393 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:10.848354 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7ddd55fbb6-94gf2"] Jul 06 23:57:13.424113 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.424081 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-xlhnc"] Jul 06 23:57:13.424530 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.424413 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b1c8289-6786-41c6-a0fd-66e613080735" containerName="util" Jul 06 23:57:13.424530 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.424425 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b1c8289-6786-41c6-a0fd-66e613080735" containerName="util" Jul 06 23:57:13.424530 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.424434 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4be0703-a52e-49d4-8942-edbb447df0ad" containerName="extract" Jul 06 23:57:13.424530 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.424440 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4be0703-a52e-49d4-8942-edbb447df0ad" containerName="extract" Jul 06 23:57:13.424530 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.424449 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4be0703-a52e-49d4-8942-edbb447df0ad" containerName="util" Jul 06 23:57:13.424530 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.424456 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4be0703-a52e-49d4-8942-edbb447df0ad" containerName="util" Jul 06 23:57:13.424530 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.424464 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4be0703-a52e-49d4-8942-edbb447df0ad" containerName="pull" Jul 06 23:57:13.424530 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.424470 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4be0703-a52e-49d4-8942-edbb447df0ad" containerName="pull" Jul 06 23:57:13.424530 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.424476 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4edb7a7-2478-43e1-b346-d786e1606487" containerName="extract" Jul 06 23:57:13.424530 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.424481 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4edb7a7-2478-43e1-b346-d786e1606487" containerName="extract" Jul 06 23:57:13.424530 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.424488 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4edb7a7-2478-43e1-b346-d786e1606487" containerName="pull" Jul 06 23:57:13.424530 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.424494 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4edb7a7-2478-43e1-b346-d786e1606487" containerName="pull" Jul 06 23:57:13.424530 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.424505 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b1c8289-6786-41c6-a0fd-66e613080735" containerName="pull" Jul 06 23:57:13.424530 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.424509 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b1c8289-6786-41c6-a0fd-66e613080735" containerName="pull" Jul 06 23:57:13.424530 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.424518 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4edb7a7-2478-43e1-b346-d786e1606487" containerName="util" Jul 06 23:57:13.424530 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.424522 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4edb7a7-2478-43e1-b346-d786e1606487" containerName="util" Jul 06 23:57:13.424530 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.424528 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3ced335-f964-46a9-b311-969a96fc1136" containerName="pull" Jul 06 23:57:13.424530 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.424533 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3ced335-f964-46a9-b311-969a96fc1136" containerName="pull" Jul 06 23:57:13.424530 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.424539 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3ced335-f964-46a9-b311-969a96fc1136" containerName="util" Jul 06 23:57:13.425137 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.424544 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3ced335-f964-46a9-b311-969a96fc1136" containerName="util" Jul 06 23:57:13.425137 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.424550 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3ced335-f964-46a9-b311-969a96fc1136" containerName="extract" Jul 06 23:57:13.425137 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.424555 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3ced335-f964-46a9-b311-969a96fc1136" containerName="extract" Jul 06 23:57:13.425137 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.424563 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b1c8289-6786-41c6-a0fd-66e613080735" containerName="extract" Jul 06 23:57:13.425137 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.424568 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b1c8289-6786-41c6-a0fd-66e613080735" containerName="extract" Jul 06 23:57:13.425137 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.424619 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4edb7a7-2478-43e1-b346-d786e1606487" containerName="extract" Jul 06 23:57:13.425137 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.424652 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3ced335-f964-46a9-b311-969a96fc1136" containerName="extract" Jul 06 23:57:13.425137 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.424659 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b1c8289-6786-41c6-a0fd-66e613080735" containerName="extract" Jul 06 23:57:13.425137 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.424665 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4be0703-a52e-49d4-8942-edbb447df0ad" containerName="extract" Jul 06 23:57:13.429621 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.429600 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-xlhnc" Jul 06 23:57:13.432197 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.432179 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-xkshr\"" Jul 06 23:57:13.437653 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.437588 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-xlhnc"] Jul 06 23:57:13.536831 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.536794 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz9nf\" (UniqueName: \"kubernetes.io/projected/a58195c7-3ab7-4748-a16f-58f9f6d1d23e-kube-api-access-nz9nf\") pod \"authorino-operator-75cc8d5cb7-xlhnc\" (UID: \"a58195c7-3ab7-4748-a16f-58f9f6d1d23e\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-xlhnc" Jul 06 23:57:13.637348 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.637316 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz9nf\" (UniqueName: \"kubernetes.io/projected/a58195c7-3ab7-4748-a16f-58f9f6d1d23e-kube-api-access-nz9nf\") pod \"authorino-operator-75cc8d5cb7-xlhnc\" (UID: \"a58195c7-3ab7-4748-a16f-58f9f6d1d23e\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-xlhnc" Jul 06 23:57:13.647096 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.647069 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz9nf\" (UniqueName: \"kubernetes.io/projected/a58195c7-3ab7-4748-a16f-58f9f6d1d23e-kube-api-access-nz9nf\") pod \"authorino-operator-75cc8d5cb7-xlhnc\" (UID: \"a58195c7-3ab7-4748-a16f-58f9f6d1d23e\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-xlhnc" Jul 06 23:57:13.741391 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.741274 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-xlhnc" Jul 06 23:57:13.905011 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:13.904811 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-xlhnc"] Jul 06 23:57:14.817573 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:14.817528 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-xlhnc" event={"ID":"a58195c7-3ab7-4748-a16f-58f9f6d1d23e","Type":"ContainerStarted","Data":"01bce2ba4ddfe787c1c63c5571b591b2faace1f17f391511411c5b4843d67cb1"} Jul 06 23:57:16.831258 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:16.831219 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-xlhnc" event={"ID":"a58195c7-3ab7-4748-a16f-58f9f6d1d23e","Type":"ContainerStarted","Data":"32fed7dc21cdb8f390b759f5a029d8717dcc0d9e79a50d145fbcdc6f859772d6"} Jul 06 23:57:16.831693 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:16.831328 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-xlhnc" Jul 06 23:57:16.854674 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:16.854621 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-xlhnc" podStartSLOduration=1.691622981 podStartE2EDuration="3.854604763s" totalImagesPullingTime="2.162981782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:16.852351017 +0000 UTC m=+519.062080558" watchObservedRunningTime="2026-07-06 23:57:16.854604763 +0000 UTC m=+519.064334284" Jul 06 23:57:18.502545 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:18.502506 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz"] Jul 06 23:57:18.506343 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:18.506316 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz" Jul 06 23:57:18.510025 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:18.510003 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hrvd6\"" Jul 06 23:57:18.530341 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:18.530306 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz"] Jul 06 23:57:18.585473 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:18.585436 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8e7cec6d-43e8-4668-b785-acbeb9a9aa47-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p6drz\" (UID: \"8e7cec6d-43e8-4668-b785-acbeb9a9aa47\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz" Jul 06 23:57:18.585671 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:18.585481 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k58fc\" (UniqueName: \"kubernetes.io/projected/8e7cec6d-43e8-4668-b785-acbeb9a9aa47-kube-api-access-k58fc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p6drz\" (UID: \"8e7cec6d-43e8-4668-b785-acbeb9a9aa47\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz" Jul 06 23:57:18.686346 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:18.686281 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8e7cec6d-43e8-4668-b785-acbeb9a9aa47-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p6drz\" (UID: \"8e7cec6d-43e8-4668-b785-acbeb9a9aa47\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz" Jul 06 23:57:18.686518 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:18.686356 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k58fc\" (UniqueName: \"kubernetes.io/projected/8e7cec6d-43e8-4668-b785-acbeb9a9aa47-kube-api-access-k58fc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p6drz\" (UID: \"8e7cec6d-43e8-4668-b785-acbeb9a9aa47\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz" Jul 06 23:57:18.686684 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:18.686662 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8e7cec6d-43e8-4668-b785-acbeb9a9aa47-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p6drz\" (UID: \"8e7cec6d-43e8-4668-b785-acbeb9a9aa47\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz" Jul 06 23:57:18.695227 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:18.695202 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k58fc\" (UniqueName: \"kubernetes.io/projected/8e7cec6d-43e8-4668-b785-acbeb9a9aa47-kube-api-access-k58fc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-p6drz\" (UID: \"8e7cec6d-43e8-4668-b785-acbeb9a9aa47\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz" Jul 06 23:57:18.816455 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:18.816350 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz" Jul 06 23:57:18.960823 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:18.960799 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz"] Jul 06 23:57:18.962967 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:57:18.962940 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e7cec6d_43e8_4668_b785_acbeb9a9aa47.slice/crio-ba6fde4c7e204da5f1fad81c220d42d4fef24ae8d01d3f3adf3c31a25ad0fa27 WatchSource:0}: Error finding container ba6fde4c7e204da5f1fad81c220d42d4fef24ae8d01d3f3adf3c31a25ad0fa27: Status 404 returned error can't find the container with id ba6fde4c7e204da5f1fad81c220d42d4fef24ae8d01d3f3adf3c31a25ad0fa27 Jul 06 23:57:19.852005 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:19.851971 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz" event={"ID":"8e7cec6d-43e8-4668-b785-acbeb9a9aa47","Type":"ContainerStarted","Data":"ba6fde4c7e204da5f1fad81c220d42d4fef24ae8d01d3f3adf3c31a25ad0fa27"} Jul 06 23:57:21.345552 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:21.345507 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-sw76w"] Jul 06 23:57:21.349289 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:21.349267 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sw76w" Jul 06 23:57:21.352220 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:21.352189 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 06 23:57:21.352369 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:21.352223 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-7mzqp\"" Jul 06 23:57:21.359574 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:21.359514 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-sw76w"] Jul 06 23:57:21.412508 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:21.412476 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzj29\" (UniqueName: \"kubernetes.io/projected/e090dac8-d5d5-4ea9-a0c2-1ffec011d9e2-kube-api-access-fzj29\") pod \"dns-operator-controller-manager-864dd78c4c-sw76w\" (UID: \"e090dac8-d5d5-4ea9-a0c2-1ffec011d9e2\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sw76w" Jul 06 23:57:21.513869 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:21.513831 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzj29\" (UniqueName: \"kubernetes.io/projected/e090dac8-d5d5-4ea9-a0c2-1ffec011d9e2-kube-api-access-fzj29\") pod \"dns-operator-controller-manager-864dd78c4c-sw76w\" (UID: \"e090dac8-d5d5-4ea9-a0c2-1ffec011d9e2\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sw76w" Jul 06 23:57:21.524516 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:21.524484 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzj29\" (UniqueName: \"kubernetes.io/projected/e090dac8-d5d5-4ea9-a0c2-1ffec011d9e2-kube-api-access-fzj29\") pod \"dns-operator-controller-manager-864dd78c4c-sw76w\" (UID: \"e090dac8-d5d5-4ea9-a0c2-1ffec011d9e2\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sw76w" Jul 06 23:57:21.663465 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:21.663431 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sw76w" Jul 06 23:57:22.966701 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:22.966670 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-sw76w"] Jul 06 23:57:22.968273 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:57:22.968247 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode090dac8_d5d5_4ea9_a0c2_1ffec011d9e2.slice/crio-3b64b76fd064b73388c693c391ee5b360fcdc39ff425781ee0abc96ef3b4d4d4 WatchSource:0}: Error finding container 3b64b76fd064b73388c693c391ee5b360fcdc39ff425781ee0abc96ef3b4d4d4: Status 404 returned error can't find the container with id 3b64b76fd064b73388c693c391ee5b360fcdc39ff425781ee0abc96ef3b4d4d4 Jul 06 23:57:23.818811 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:23.818774 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k"] Jul 06 23:57:23.824028 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:23.824000 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k" Jul 06 23:57:23.827409 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:23.827373 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-jv28c\"" Jul 06 23:57:23.831924 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:23.831898 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k"] Jul 06 23:57:23.879251 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:23.879205 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz" event={"ID":"8e7cec6d-43e8-4668-b785-acbeb9a9aa47","Type":"ContainerStarted","Data":"263c2c2479a8646e85017bfb07cf6de46b1807e556af311a2215f1b57e4fc2f2"} Jul 06 23:57:23.879445 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:23.879321 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz" Jul 06 23:57:23.881094 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:23.881065 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sw76w" event={"ID":"e090dac8-d5d5-4ea9-a0c2-1ffec011d9e2","Type":"ContainerStarted","Data":"3b64b76fd064b73388c693c391ee5b360fcdc39ff425781ee0abc96ef3b4d4d4"} Jul 06 23:57:23.902032 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:23.901892 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz" podStartSLOduration=1.983154493 podStartE2EDuration="5.901870655s" totalImagesPullingTime="3.918716162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:23.901612364 +0000 UTC m=+526.111341906" watchObservedRunningTime="2026-07-06 23:57:23.901870655 +0000 UTC m=+526.111600169" Jul 06 23:57:23.937967 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:23.937932 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtl24\" (UniqueName: \"kubernetes.io/projected/48453282-83d3-4233-83e5-1943bc2070f7-kube-api-access-vtl24\") pod \"limitador-operator-controller-manager-859d7b78bd-mgb8k\" (UID: \"48453282-83d3-4233-83e5-1943bc2070f7\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k" Jul 06 23:57:24.038997 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:24.038952 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtl24\" (UniqueName: \"kubernetes.io/projected/48453282-83d3-4233-83e5-1943bc2070f7-kube-api-access-vtl24\") pod \"limitador-operator-controller-manager-859d7b78bd-mgb8k\" (UID: \"48453282-83d3-4233-83e5-1943bc2070f7\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k" Jul 06 23:57:24.052977 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:24.052947 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtl24\" (UniqueName: \"kubernetes.io/projected/48453282-83d3-4233-83e5-1943bc2070f7-kube-api-access-vtl24\") pod \"limitador-operator-controller-manager-859d7b78bd-mgb8k\" (UID: \"48453282-83d3-4233-83e5-1943bc2070f7\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k" Jul 06 23:57:24.139156 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:24.139075 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k" Jul 06 23:57:24.406210 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:24.406186 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k"] Jul 06 23:57:24.669470 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:57:24.669383 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48453282_83d3_4233_83e5_1943bc2070f7.slice/crio-5c958960b1ce067ea7dafad5c56f185cdf3bfed085c236270e4e69d8f408da38 WatchSource:0}: Error finding container 5c958960b1ce067ea7dafad5c56f185cdf3bfed085c236270e4e69d8f408da38: Status 404 returned error can't find the container with id 5c958960b1ce067ea7dafad5c56f185cdf3bfed085c236270e4e69d8f408da38 Jul 06 23:57:24.887822 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:24.887784 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sw76w" event={"ID":"e090dac8-d5d5-4ea9-a0c2-1ffec011d9e2","Type":"ContainerStarted","Data":"ced363f1c9d1b1228bbb1b5acf49f9666edc132b7ed9d89678f52ec1a34fabd7"} Jul 06 23:57:24.888010 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:24.887857 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sw76w" Jul 06 23:57:24.889025 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:24.889002 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k" event={"ID":"48453282-83d3-4233-83e5-1943bc2070f7","Type":"ContainerStarted","Data":"5c958960b1ce067ea7dafad5c56f185cdf3bfed085c236270e4e69d8f408da38"} Jul 06 23:57:24.905891 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:24.905843 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sw76w" podStartSLOduration=2.157841225 podStartE2EDuration="3.905827526s" totalImagesPullingTime="1.747986301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:24.903780274 +0000 UTC m=+527.113509815" watchObservedRunningTime="2026-07-06 23:57:24.905827526 +0000 UTC m=+527.115557047" Jul 06 23:57:26.901914 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:26.901873 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k" event={"ID":"48453282-83d3-4233-83e5-1943bc2070f7","Type":"ContainerStarted","Data":"e9aba4a9c2dea736837abca767b71d481756b45a6a6b84764557f8e8975bb2dd"} Jul 06 23:57:26.902371 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:26.901947 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k" Jul 06 23:57:26.921983 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:26.921927 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k" podStartSLOduration=2.656857311 podStartE2EDuration="3.921910006s" totalImagesPullingTime="1.265052695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:26.91981196 +0000 UTC m=+529.129541509" watchObservedRunningTime="2026-07-06 23:57:26.921910006 +0000 UTC m=+529.131639583" Jul 06 23:57:27.839382 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:27.839347 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-xlhnc" Jul 06 23:57:34.891318 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:34.891216 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz" Jul 06 23:57:35.876347 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:35.876289 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7ddd55fbb6-94gf2" podUID="4f108968-7328-4771-92b3-5db9e1a812dc" containerName="console" containerID="cri-o://75c6506bd9e7916874249ff6e6b348fa8f10f302c95fc4497d5ce2904606508c" gracePeriod=15 Jul 06 23:57:35.897728 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:35.897695 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-sw76w" Jul 06 23:57:36.131153 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.131083 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ddd55fbb6-94gf2" Jul 06 23:57:36.245450 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.245417 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f108968-7328-4771-92b3-5db9e1a812dc-trusted-ca-bundle\") pod \"4f108968-7328-4771-92b3-5db9e1a812dc\" (UID: \"4f108968-7328-4771-92b3-5db9e1a812dc\") " Jul 06 23:57:36.245650 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.245469 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f108968-7328-4771-92b3-5db9e1a812dc-console-serving-cert\") pod \"4f108968-7328-4771-92b3-5db9e1a812dc\" (UID: \"4f108968-7328-4771-92b3-5db9e1a812dc\") " Jul 06 23:57:36.245650 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.245525 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f108968-7328-4771-92b3-5db9e1a812dc-service-ca\") pod \"4f108968-7328-4771-92b3-5db9e1a812dc\" (UID: \"4f108968-7328-4771-92b3-5db9e1a812dc\") " Jul 06 23:57:36.245650 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.245542 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f108968-7328-4771-92b3-5db9e1a812dc-console-oauth-config\") pod \"4f108968-7328-4771-92b3-5db9e1a812dc\" (UID: \"4f108968-7328-4771-92b3-5db9e1a812dc\") " Jul 06 23:57:36.245650 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.245578 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kn4fx\" (UniqueName: \"kubernetes.io/projected/4f108968-7328-4771-92b3-5db9e1a812dc-kube-api-access-kn4fx\") pod \"4f108968-7328-4771-92b3-5db9e1a812dc\" (UID: \"4f108968-7328-4771-92b3-5db9e1a812dc\") " Jul 06 23:57:36.245650 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.245631 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f108968-7328-4771-92b3-5db9e1a812dc-oauth-serving-cert\") pod \"4f108968-7328-4771-92b3-5db9e1a812dc\" (UID: \"4f108968-7328-4771-92b3-5db9e1a812dc\") " Jul 06 23:57:36.245919 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.245658 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f108968-7328-4771-92b3-5db9e1a812dc-console-config\") pod \"4f108968-7328-4771-92b3-5db9e1a812dc\" (UID: \"4f108968-7328-4771-92b3-5db9e1a812dc\") " Jul 06 23:57:36.246056 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.246025 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f108968-7328-4771-92b3-5db9e1a812dc-service-ca" (OuterVolumeSpecName: "service-ca") pod "4f108968-7328-4771-92b3-5db9e1a812dc" (UID: "4f108968-7328-4771-92b3-5db9e1a812dc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:57:36.246131 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.246077 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f108968-7328-4771-92b3-5db9e1a812dc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4f108968-7328-4771-92b3-5db9e1a812dc" (UID: "4f108968-7328-4771-92b3-5db9e1a812dc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:57:36.246257 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.246232 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f108968-7328-4771-92b3-5db9e1a812dc-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4f108968-7328-4771-92b3-5db9e1a812dc" (UID: "4f108968-7328-4771-92b3-5db9e1a812dc"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:57:36.246348 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.246259 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f108968-7328-4771-92b3-5db9e1a812dc-console-config" (OuterVolumeSpecName: "console-config") pod "4f108968-7328-4771-92b3-5db9e1a812dc" (UID: "4f108968-7328-4771-92b3-5db9e1a812dc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:57:36.247804 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.247778 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f108968-7328-4771-92b3-5db9e1a812dc-kube-api-access-kn4fx" (OuterVolumeSpecName: "kube-api-access-kn4fx") pod "4f108968-7328-4771-92b3-5db9e1a812dc" (UID: "4f108968-7328-4771-92b3-5db9e1a812dc"). InnerVolumeSpecName "kube-api-access-kn4fx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:57:36.247929 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.247912 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f108968-7328-4771-92b3-5db9e1a812dc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4f108968-7328-4771-92b3-5db9e1a812dc" (UID: "4f108968-7328-4771-92b3-5db9e1a812dc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:57:36.247987 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.247939 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f108968-7328-4771-92b3-5db9e1a812dc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4f108968-7328-4771-92b3-5db9e1a812dc" (UID: "4f108968-7328-4771-92b3-5db9e1a812dc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:57:36.347332 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.347266 2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f108968-7328-4771-92b3-5db9e1a812dc-service-ca\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:57:36.347332 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.347322 2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f108968-7328-4771-92b3-5db9e1a812dc-console-oauth-config\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:57:36.347332 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.347339 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kn4fx\" (UniqueName: \"kubernetes.io/projected/4f108968-7328-4771-92b3-5db9e1a812dc-kube-api-access-kn4fx\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:57:36.347583 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.347353 2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f108968-7328-4771-92b3-5db9e1a812dc-oauth-serving-cert\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:57:36.347583 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.347367 2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f108968-7328-4771-92b3-5db9e1a812dc-console-config\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:57:36.347583 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.347376 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f108968-7328-4771-92b3-5db9e1a812dc-trusted-ca-bundle\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:57:36.347583 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.347385 2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f108968-7328-4771-92b3-5db9e1a812dc-console-serving-cert\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:57:36.427842 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.427802 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4vtwn"] Jul 06 23:57:36.428203 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.428190 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f108968-7328-4771-92b3-5db9e1a812dc" containerName="console" Jul 06 23:57:36.428267 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.428205 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f108968-7328-4771-92b3-5db9e1a812dc" containerName="console" Jul 06 23:57:36.428351 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.428274 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f108968-7328-4771-92b3-5db9e1a812dc" containerName="console" Jul 06 23:57:36.431732 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.431715 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4vtwn" Jul 06 23:57:36.444678 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.444636 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4vtwn"] Jul 06 23:57:36.549276 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.549230 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/657f1aa7-dd12-406f-a8ea-4084e434eb01-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-4vtwn\" (UID: \"657f1aa7-dd12-406f-a8ea-4084e434eb01\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4vtwn" Jul 06 23:57:36.549479 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.549365 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj8vz\" (UniqueName: \"kubernetes.io/projected/657f1aa7-dd12-406f-a8ea-4084e434eb01-kube-api-access-jj8vz\") pod \"kuadrant-operator-controller-manager-55467d9f7-4vtwn\" (UID: \"657f1aa7-dd12-406f-a8ea-4084e434eb01\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4vtwn" Jul 06 23:57:36.618585 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.618555 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4vtwn"] Jul 06 23:57:36.618880 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:36.618857 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-jj8vz], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4vtwn" podUID="657f1aa7-dd12-406f-a8ea-4084e434eb01" Jul 06 23:57:36.629535 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.629507 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4vtwn"] Jul 06 23:57:36.645448 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.645406 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l7fhj"] Jul 06 23:57:36.648882 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.648859 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz"] Jul 06 23:57:36.649011 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.648944 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l7fhj" Jul 06 23:57:36.649085 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.649027 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz" podUID="8e7cec6d-43e8-4668-b785-acbeb9a9aa47" containerName="manager" containerID="cri-o://263c2c2479a8646e85017bfb07cf6de46b1807e556af311a2215f1b57e4fc2f2" gracePeriod=2 Jul 06 23:57:36.650566 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.650509 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/657f1aa7-dd12-406f-a8ea-4084e434eb01-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-4vtwn\" (UID: \"657f1aa7-dd12-406f-a8ea-4084e434eb01\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4vtwn" Jul 06 23:57:36.650688 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.650628 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj8vz\" (UniqueName: \"kubernetes.io/projected/657f1aa7-dd12-406f-a8ea-4084e434eb01-kube-api-access-jj8vz\") pod \"kuadrant-operator-controller-manager-55467d9f7-4vtwn\" (UID: \"657f1aa7-dd12-406f-a8ea-4084e434eb01\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4vtwn" Jul 06 23:57:36.650901 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.650880 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/657f1aa7-dd12-406f-a8ea-4084e434eb01-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-4vtwn\" (UID: \"657f1aa7-dd12-406f-a8ea-4084e434eb01\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4vtwn" Jul 06 23:57:36.655659 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:36.655622 2553 projected.go:196] Error preparing data for projected volume kube-api-access-jj8vz for pod kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4vtwn: failed to fetch token: pod "kuadrant-operator-controller-manager-55467d9f7-4vtwn" not found Jul 06 23:57:36.656466 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:36.656446 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/657f1aa7-dd12-406f-a8ea-4084e434eb01-kube-api-access-jj8vz podName:657f1aa7-dd12-406f-a8ea-4084e434eb01 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:37.156425532 +0000 UTC m=+539.366155044 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jj8vz" (UniqueName: "kubernetes.io/projected/657f1aa7-dd12-406f-a8ea-4084e434eb01-kube-api-access-jj8vz") pod "kuadrant-operator-controller-manager-55467d9f7-4vtwn" (UID: "657f1aa7-dd12-406f-a8ea-4084e434eb01") : failed to fetch token: pod "kuadrant-operator-controller-manager-55467d9f7-4vtwn" not found Jul 06 23:57:36.659848 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.659676 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz"] Jul 06 23:57:36.665785 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:36.665745 2553 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"8e7cec6d-43e8-4668-b785-acbeb9a9aa47\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-p6drz\": pods \"kuadrant-operator-controller-manager-8684c9ff49-p6drz\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz" Jul 06 23:57:36.667055 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.667034 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l7fhj"] Jul 06 23:57:36.672843 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.672819 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k"] Jul 06 23:57:36.673041 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.673015 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k" podUID="48453282-83d3-4233-83e5-1943bc2070f7" containerName="manager" containerID="cri-o://e9aba4a9c2dea736837abca767b71d481756b45a6a6b84764557f8e8975bb2dd" gracePeriod=2 Jul 06 23:57:36.674748 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.674726 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k" Jul 06 23:57:36.680811 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.680718 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wm4s6"] Jul 06 23:57:36.681321 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.681244 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e7cec6d-43e8-4668-b785-acbeb9a9aa47" containerName="manager" Jul 06 23:57:36.681321 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.681268 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e7cec6d-43e8-4668-b785-acbeb9a9aa47" containerName="manager" Jul 06 23:57:36.681752 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.681395 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e7cec6d-43e8-4668-b785-acbeb9a9aa47" containerName="manager" Jul 06 23:57:36.684711 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.684690 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l7fhj"] Jul 06 23:57:36.684882 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.684860 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wm4s6" Jul 06 23:57:36.685028 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:36.685006 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-xx22r], unattached volumes=[], failed to process volumes=[extensions-socket-volume kube-api-access-xx22r]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l7fhj" podUID="e2c92859-c311-4cf3-a145-8adb24b99b25" Jul 06 23:57:36.687115 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.687092 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7nh2l"] Jul 06 23:57:36.690919 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.690899 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k"] Jul 06 23:57:36.691008 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.690990 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7nh2l" Jul 06 23:57:36.701173 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.701136 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wm4s6"] Jul 06 23:57:36.703643 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.703621 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7nh2l"] Jul 06 23:57:36.717839 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:36.717800 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p6drz\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="8e7cec6d-43e8-4668-b785-acbeb9a9aa47" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz" Jul 06 23:57:36.718573 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.718554 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d8fs6"] Jul 06 23:57:36.719119 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.719103 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48453282-83d3-4233-83e5-1943bc2070f7" containerName="manager" Jul 06 23:57:36.719207 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.719124 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="48453282-83d3-4233-83e5-1943bc2070f7" containerName="manager" Jul 06 23:57:36.719269 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.719225 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="48453282-83d3-4233-83e5-1943bc2070f7" containerName="manager" Jul 06 23:57:36.722498 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.722476 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d8fs6" Jul 06 23:57:36.734743 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.734719 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d8fs6"] Jul 06 23:57:36.751641 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.751601 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa56e287-b843-4b0f-b49b-cec8ba10a6de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wm4s6\" (UID: \"fa56e287-b843-4b0f-b49b-cec8ba10a6de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wm4s6" Jul 06 23:57:36.751826 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.751670 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e2c92859-c311-4cf3-a145-8adb24b99b25-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-l7fhj\" (UID: \"e2c92859-c311-4cf3-a145-8adb24b99b25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l7fhj" Jul 06 23:57:36.751826 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.751696 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx22r\" (UniqueName: \"kubernetes.io/projected/e2c92859-c311-4cf3-a145-8adb24b99b25-kube-api-access-xx22r\") pod \"kuadrant-operator-controller-manager-55467d9f7-l7fhj\" (UID: \"e2c92859-c311-4cf3-a145-8adb24b99b25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l7fhj" Jul 06 23:57:36.751826 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.751752 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb7zz\" (UniqueName: \"kubernetes.io/projected/fa56e287-b843-4b0f-b49b-cec8ba10a6de-kube-api-access-cb7zz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wm4s6\" (UID: \"fa56e287-b843-4b0f-b49b-cec8ba10a6de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wm4s6" Jul 06 23:57:36.765607 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:36.765566 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mgb8k\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="48453282-83d3-4233-83e5-1943bc2070f7" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k" Jul 06 23:57:36.767844 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:36.767803 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mgb8k\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="48453282-83d3-4233-83e5-1943bc2070f7" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k" Jul 06 23:57:36.791975 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:36.791933 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p6drz\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="8e7cec6d-43e8-4668-b785-acbeb9a9aa47" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz" Jul 06 23:57:36.853061 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.853030 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/92e2d5ca-67d5-46e1-bd03-d236964fe689-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7nh2l\" (UID: \"92e2d5ca-67d5-46e1-bd03-d236964fe689\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7nh2l" Jul 06 23:57:36.853186 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.853120 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa56e287-b843-4b0f-b49b-cec8ba10a6de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wm4s6\" (UID: \"fa56e287-b843-4b0f-b49b-cec8ba10a6de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wm4s6" Jul 06 23:57:36.853186 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.853177 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e2c92859-c311-4cf3-a145-8adb24b99b25-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-l7fhj\" (UID: \"e2c92859-c311-4cf3-a145-8adb24b99b25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l7fhj" Jul 06 23:57:36.853333 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.853204 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx22r\" (UniqueName: \"kubernetes.io/projected/e2c92859-c311-4cf3-a145-8adb24b99b25-kube-api-access-xx22r\") pod \"kuadrant-operator-controller-manager-55467d9f7-l7fhj\" (UID: \"e2c92859-c311-4cf3-a145-8adb24b99b25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l7fhj" Jul 06 23:57:36.853333 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.853262 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb7zz\" (UniqueName: \"kubernetes.io/projected/fa56e287-b843-4b0f-b49b-cec8ba10a6de-kube-api-access-cb7zz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wm4s6\" (UID: \"fa56e287-b843-4b0f-b49b-cec8ba10a6de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wm4s6" Jul 06 23:57:36.853333 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.853312 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d8j7\" (UniqueName: \"kubernetes.io/projected/ee864a87-f13d-418c-9f4e-43dc12ce6add-kube-api-access-9d8j7\") pod \"limitador-operator-controller-manager-859d7b78bd-d8fs6\" (UID: \"ee864a87-f13d-418c-9f4e-43dc12ce6add\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d8fs6" Jul 06 23:57:36.853570 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.853361 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjvph\" (UniqueName: \"kubernetes.io/projected/92e2d5ca-67d5-46e1-bd03-d236964fe689-kube-api-access-tjvph\") pod \"kuadrant-operator-controller-manager-86878c769f-7nh2l\" (UID: \"92e2d5ca-67d5-46e1-bd03-d236964fe689\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7nh2l" Jul 06 23:57:36.853690 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.853634 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e2c92859-c311-4cf3-a145-8adb24b99b25-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-l7fhj\" (UID: \"e2c92859-c311-4cf3-a145-8adb24b99b25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l7fhj" Jul 06 23:57:36.854901 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.854871 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa56e287-b843-4b0f-b49b-cec8ba10a6de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wm4s6\" (UID: \"fa56e287-b843-4b0f-b49b-cec8ba10a6de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wm4s6" Jul 06 23:57:36.863400 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.863376 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb7zz\" (UniqueName: \"kubernetes.io/projected/fa56e287-b843-4b0f-b49b-cec8ba10a6de-kube-api-access-cb7zz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wm4s6\" (UID: \"fa56e287-b843-4b0f-b49b-cec8ba10a6de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wm4s6" Jul 06 23:57:36.863505 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.863461 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx22r\" (UniqueName: \"kubernetes.io/projected/e2c92859-c311-4cf3-a145-8adb24b99b25-kube-api-access-xx22r\") pod \"kuadrant-operator-controller-manager-55467d9f7-l7fhj\" (UID: \"e2c92859-c311-4cf3-a145-8adb24b99b25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l7fhj" Jul 06 23:57:36.923330 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.923275 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz" Jul 06 23:57:36.925391 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.925373 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k" Jul 06 23:57:36.925805 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:36.925783 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mgb8k\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="48453282-83d3-4233-83e5-1943bc2070f7" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k" Jul 06 23:57:36.927727 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:36.927706 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p6drz\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="8e7cec6d-43e8-4668-b785-acbeb9a9aa47" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz" Jul 06 23:57:36.929741 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:36.929719 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mgb8k\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="48453282-83d3-4233-83e5-1943bc2070f7" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k" Jul 06 23:57:36.931622 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:36.931580 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p6drz\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="8e7cec6d-43e8-4668-b785-acbeb9a9aa47" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz" Jul 06 23:57:36.953760 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.953738 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjvph\" (UniqueName: \"kubernetes.io/projected/92e2d5ca-67d5-46e1-bd03-d236964fe689-kube-api-access-tjvph\") pod \"kuadrant-operator-controller-manager-86878c769f-7nh2l\" (UID: \"92e2d5ca-67d5-46e1-bd03-d236964fe689\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7nh2l" Jul 06 23:57:36.953916 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.953797 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/92e2d5ca-67d5-46e1-bd03-d236964fe689-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7nh2l\" (UID: \"92e2d5ca-67d5-46e1-bd03-d236964fe689\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7nh2l" Jul 06 23:57:36.953916 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.953897 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d8j7\" (UniqueName: \"kubernetes.io/projected/ee864a87-f13d-418c-9f4e-43dc12ce6add-kube-api-access-9d8j7\") pod \"limitador-operator-controller-manager-859d7b78bd-d8fs6\" (UID: \"ee864a87-f13d-418c-9f4e-43dc12ce6add\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d8fs6" Jul 06 23:57:36.954192 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.954163 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/92e2d5ca-67d5-46e1-bd03-d236964fe689-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7nh2l\" (UID: \"92e2d5ca-67d5-46e1-bd03-d236964fe689\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7nh2l" Jul 06 23:57:36.954785 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.954763 2553 generic.go:356] "Generic (PLEG): container finished" podID="4f108968-7328-4771-92b3-5db9e1a812dc" containerID="75c6506bd9e7916874249ff6e6b348fa8f10f302c95fc4497d5ce2904606508c" exitCode=0 Jul 06 23:57:36.954881 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.954814 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ddd55fbb6-94gf2" Jul 06 23:57:36.954881 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.954846 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ddd55fbb6-94gf2" event={"ID":"4f108968-7328-4771-92b3-5db9e1a812dc","Type":"ContainerDied","Data":"75c6506bd9e7916874249ff6e6b348fa8f10f302c95fc4497d5ce2904606508c"} Jul 06 23:57:36.954990 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.954897 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ddd55fbb6-94gf2" event={"ID":"4f108968-7328-4771-92b3-5db9e1a812dc","Type":"ContainerDied","Data":"360fb2c24d9579b76d523027b450200a98af6b523ce5b8965111bf3558370f5f"} Jul 06 23:57:36.954990 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.954919 2553 scope.go:117] "RemoveContainer" containerID="75c6506bd9e7916874249ff6e6b348fa8f10f302c95fc4497d5ce2904606508c" Jul 06 23:57:36.956244 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.956221 2553 generic.go:356] "Generic (PLEG): container finished" podID="8e7cec6d-43e8-4668-b785-acbeb9a9aa47" containerID="263c2c2479a8646e85017bfb07cf6de46b1807e556af311a2215f1b57e4fc2f2" exitCode=0 Jul 06 23:57:36.956351 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.956272 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz" Jul 06 23:57:36.958018 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.957994 2553 generic.go:356] "Generic (PLEG): container finished" podID="48453282-83d3-4233-83e5-1943bc2070f7" containerID="e9aba4a9c2dea736837abca767b71d481756b45a6a6b84764557f8e8975bb2dd" exitCode=0 Jul 06 23:57:36.958115 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.958027 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k" Jul 06 23:57:36.958180 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.958153 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l7fhj" Jul 06 23:57:36.958239 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.958180 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4vtwn" Jul 06 23:57:36.962668 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:36.962641 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p6drz\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="8e7cec6d-43e8-4668-b785-acbeb9a9aa47" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz" Jul 06 23:57:36.965078 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:36.965011 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mgb8k\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="48453282-83d3-4233-83e5-1943bc2070f7" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k" Jul 06 23:57:36.965185 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.965136 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d8j7\" (UniqueName: \"kubernetes.io/projected/ee864a87-f13d-418c-9f4e-43dc12ce6add-kube-api-access-9d8j7\") pod \"limitador-operator-controller-manager-859d7b78bd-d8fs6\" (UID: \"ee864a87-f13d-418c-9f4e-43dc12ce6add\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d8fs6" Jul 06 23:57:36.965249 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.965197 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l7fhj" Jul 06 23:57:36.965467 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.965450 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjvph\" (UniqueName: \"kubernetes.io/projected/92e2d5ca-67d5-46e1-bd03-d236964fe689-kube-api-access-tjvph\") pod \"kuadrant-operator-controller-manager-86878c769f-7nh2l\" (UID: \"92e2d5ca-67d5-46e1-bd03-d236964fe689\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7nh2l" Jul 06 23:57:36.966019 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.966001 2553 scope.go:117] "RemoveContainer" containerID="75c6506bd9e7916874249ff6e6b348fa8f10f302c95fc4497d5ce2904606508c" Jul 06 23:57:36.966282 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:36.966264 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75c6506bd9e7916874249ff6e6b348fa8f10f302c95fc4497d5ce2904606508c\": container with ID starting with 75c6506bd9e7916874249ff6e6b348fa8f10f302c95fc4497d5ce2904606508c not found: ID does not exist" containerID="75c6506bd9e7916874249ff6e6b348fa8f10f302c95fc4497d5ce2904606508c" Jul 06 23:57:36.966386 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.966290 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75c6506bd9e7916874249ff6e6b348fa8f10f302c95fc4497d5ce2904606508c"} err="failed to get container status \"75c6506bd9e7916874249ff6e6b348fa8f10f302c95fc4497d5ce2904606508c\": rpc error: code = NotFound desc = could not find container \"75c6506bd9e7916874249ff6e6b348fa8f10f302c95fc4497d5ce2904606508c\": container with ID starting with 75c6506bd9e7916874249ff6e6b348fa8f10f302c95fc4497d5ce2904606508c not found: ID does not exist" Jul 06 23:57:36.966386 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.966329 2553 scope.go:117] "RemoveContainer" containerID="263c2c2479a8646e85017bfb07cf6de46b1807e556af311a2215f1b57e4fc2f2" Jul 06 23:57:36.977845 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.977826 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4vtwn" Jul 06 23:57:36.978219 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.978206 2553 scope.go:117] "RemoveContainer" containerID="263c2c2479a8646e85017bfb07cf6de46b1807e556af311a2215f1b57e4fc2f2" Jul 06 23:57:36.978491 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:36.978473 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"263c2c2479a8646e85017bfb07cf6de46b1807e556af311a2215f1b57e4fc2f2\": container with ID starting with 263c2c2479a8646e85017bfb07cf6de46b1807e556af311a2215f1b57e4fc2f2 not found: ID does not exist" containerID="263c2c2479a8646e85017bfb07cf6de46b1807e556af311a2215f1b57e4fc2f2" Jul 06 23:57:36.978538 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.978497 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"263c2c2479a8646e85017bfb07cf6de46b1807e556af311a2215f1b57e4fc2f2"} err="failed to get container status \"263c2c2479a8646e85017bfb07cf6de46b1807e556af311a2215f1b57e4fc2f2\": rpc error: code = NotFound desc = could not find container \"263c2c2479a8646e85017bfb07cf6de46b1807e556af311a2215f1b57e4fc2f2\": container with ID starting with 263c2c2479a8646e85017bfb07cf6de46b1807e556af311a2215f1b57e4fc2f2 not found: ID does not exist" Jul 06 23:57:36.978538 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.978515 2553 scope.go:117] "RemoveContainer" containerID="e9aba4a9c2dea736837abca767b71d481756b45a6a6b84764557f8e8975bb2dd" Jul 06 23:57:36.986205 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.986191 2553 scope.go:117] "RemoveContainer" containerID="e9aba4a9c2dea736837abca767b71d481756b45a6a6b84764557f8e8975bb2dd" Jul 06 23:57:36.986458 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:36.986436 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9aba4a9c2dea736837abca767b71d481756b45a6a6b84764557f8e8975bb2dd\": container with ID starting with e9aba4a9c2dea736837abca767b71d481756b45a6a6b84764557f8e8975bb2dd not found: ID does not exist" containerID="e9aba4a9c2dea736837abca767b71d481756b45a6a6b84764557f8e8975bb2dd" Jul 06 23:57:36.986531 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:36.986464 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9aba4a9c2dea736837abca767b71d481756b45a6a6b84764557f8e8975bb2dd"} err="failed to get container status \"e9aba4a9c2dea736837abca767b71d481756b45a6a6b84764557f8e8975bb2dd\": rpc error: code = NotFound desc = could not find container \"e9aba4a9c2dea736837abca767b71d481756b45a6a6b84764557f8e8975bb2dd\": container with ID starting with e9aba4a9c2dea736837abca767b71d481756b45a6a6b84764557f8e8975bb2dd not found: ID does not exist" Jul 06 23:57:36.992338 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:36.992281 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mgb8k\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="48453282-83d3-4233-83e5-1943bc2070f7" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k" Jul 06 23:57:37.016361 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.016328 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7ddd55fbb6-94gf2"] Jul 06 23:57:37.023514 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:37.023487 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p6drz\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="8e7cec6d-43e8-4668-b785-acbeb9a9aa47" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz" Jul 06 23:57:37.023859 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.023830 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7ddd55fbb6-94gf2"] Jul 06 23:57:37.025625 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:37.025603 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-4vtwn\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="657f1aa7-dd12-406f-a8ea-4084e434eb01" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4vtwn" Jul 06 23:57:37.027771 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:37.027742 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mgb8k\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="48453282-83d3-4233-83e5-1943bc2070f7" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k" Jul 06 23:57:37.029683 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:37.029665 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p6drz\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="8e7cec6d-43e8-4668-b785-acbeb9a9aa47" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz" Jul 06 23:57:37.031641 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:37.031623 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-4vtwn\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="657f1aa7-dd12-406f-a8ea-4084e434eb01" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4vtwn" Jul 06 23:57:37.054445 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.054413 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xx22r\" (UniqueName: \"kubernetes.io/projected/e2c92859-c311-4cf3-a145-8adb24b99b25-kube-api-access-xx22r\") pod \"e2c92859-c311-4cf3-a145-8adb24b99b25\" (UID: \"e2c92859-c311-4cf3-a145-8adb24b99b25\") " Jul 06 23:57:37.054596 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.054453 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8e7cec6d-43e8-4668-b785-acbeb9a9aa47-extensions-socket-volume\") pod \"8e7cec6d-43e8-4668-b785-acbeb9a9aa47\" (UID: \"8e7cec6d-43e8-4668-b785-acbeb9a9aa47\") " Jul 06 23:57:37.054596 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.054508 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e2c92859-c311-4cf3-a145-8adb24b99b25-extensions-socket-volume\") pod \"e2c92859-c311-4cf3-a145-8adb24b99b25\" (UID: \"e2c92859-c311-4cf3-a145-8adb24b99b25\") " Jul 06 23:57:37.054596 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.054555 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k58fc\" (UniqueName: \"kubernetes.io/projected/8e7cec6d-43e8-4668-b785-acbeb9a9aa47-kube-api-access-k58fc\") pod \"8e7cec6d-43e8-4668-b785-acbeb9a9aa47\" (UID: \"8e7cec6d-43e8-4668-b785-acbeb9a9aa47\") " Jul 06 23:57:37.054596 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.054584 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/657f1aa7-dd12-406f-a8ea-4084e434eb01-extensions-socket-volume\") pod \"657f1aa7-dd12-406f-a8ea-4084e434eb01\" (UID: \"657f1aa7-dd12-406f-a8ea-4084e434eb01\") " Jul 06 23:57:37.054812 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.054622 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vtl24\" (UniqueName: \"kubernetes.io/projected/48453282-83d3-4233-83e5-1943bc2070f7-kube-api-access-vtl24\") pod \"48453282-83d3-4233-83e5-1943bc2070f7\" (UID: \"48453282-83d3-4233-83e5-1943bc2070f7\") " Jul 06 23:57:37.054863 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.054812 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2c92859-c311-4cf3-a145-8adb24b99b25-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e2c92859-c311-4cf3-a145-8adb24b99b25" (UID: "e2c92859-c311-4cf3-a145-8adb24b99b25"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:57:37.054863 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.054826 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jj8vz\" (UniqueName: \"kubernetes.io/projected/657f1aa7-dd12-406f-a8ea-4084e434eb01-kube-api-access-jj8vz\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:57:37.054863 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.054829 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/657f1aa7-dd12-406f-a8ea-4084e434eb01-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "657f1aa7-dd12-406f-a8ea-4084e434eb01" (UID: "657f1aa7-dd12-406f-a8ea-4084e434eb01"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:57:37.055102 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.055079 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e7cec6d-43e8-4668-b785-acbeb9a9aa47-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8e7cec6d-43e8-4668-b785-acbeb9a9aa47" (UID: "8e7cec6d-43e8-4668-b785-acbeb9a9aa47"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:57:37.056664 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.056640 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48453282-83d3-4233-83e5-1943bc2070f7-kube-api-access-vtl24" (OuterVolumeSpecName: "kube-api-access-vtl24") pod "48453282-83d3-4233-83e5-1943bc2070f7" (UID: "48453282-83d3-4233-83e5-1943bc2070f7"). InnerVolumeSpecName "kube-api-access-vtl24". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:57:37.056736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.056678 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e7cec6d-43e8-4668-b785-acbeb9a9aa47-kube-api-access-k58fc" (OuterVolumeSpecName: "kube-api-access-k58fc") pod "8e7cec6d-43e8-4668-b785-acbeb9a9aa47" (UID: "8e7cec6d-43e8-4668-b785-acbeb9a9aa47"). InnerVolumeSpecName "kube-api-access-k58fc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:57:37.056736 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.056693 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2c92859-c311-4cf3-a145-8adb24b99b25-kube-api-access-xx22r" (OuterVolumeSpecName: "kube-api-access-xx22r") pod "e2c92859-c311-4cf3-a145-8adb24b99b25" (UID: "e2c92859-c311-4cf3-a145-8adb24b99b25"). InnerVolumeSpecName "kube-api-access-xx22r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:57:37.068901 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.068872 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wm4s6" Jul 06 23:57:37.076191 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.076168 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7nh2l" Jul 06 23:57:37.084256 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.084229 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d8fs6" Jul 06 23:57:37.156411 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.156253 2553 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e2c92859-c311-4cf3-a145-8adb24b99b25-extensions-socket-volume\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:57:37.156619 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.156547 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k58fc\" (UniqueName: \"kubernetes.io/projected/8e7cec6d-43e8-4668-b785-acbeb9a9aa47-kube-api-access-k58fc\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:57:37.156619 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.156573 2553 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/657f1aa7-dd12-406f-a8ea-4084e434eb01-extensions-socket-volume\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:57:37.156619 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.156583 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vtl24\" (UniqueName: \"kubernetes.io/projected/48453282-83d3-4233-83e5-1943bc2070f7-kube-api-access-vtl24\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:57:37.156619 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.156593 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xx22r\" (UniqueName: \"kubernetes.io/projected/e2c92859-c311-4cf3-a145-8adb24b99b25-kube-api-access-xx22r\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:57:37.156619 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.156604 2553 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8e7cec6d-43e8-4668-b785-acbeb9a9aa47-extensions-socket-volume\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:57:37.234564 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.234537 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wm4s6"] Jul 06 23:57:37.236143 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:57:37.236117 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa56e287_b843_4b0f_b49b_cec8ba10a6de.slice/crio-b4769f1195ebca0fe5de849492e54ed156f0dbe0c91146e7d95c776c52ef17bd WatchSource:0}: Error finding container b4769f1195ebca0fe5de849492e54ed156f0dbe0c91146e7d95c776c52ef17bd: Status 404 returned error can't find the container with id b4769f1195ebca0fe5de849492e54ed156f0dbe0c91146e7d95c776c52ef17bd Jul 06 23:57:37.277067 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:37.277040 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p6drz\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="8e7cec6d-43e8-4668-b785-acbeb9a9aa47" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz" Jul 06 23:57:37.279117 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:37.279059 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-4vtwn\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="657f1aa7-dd12-406f-a8ea-4084e434eb01" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4vtwn" Jul 06 23:57:37.281317 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:37.281280 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mgb8k\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="48453282-83d3-4233-83e5-1943bc2070f7" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k" Jul 06 23:57:37.283285 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:37.283266 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-4vtwn\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="657f1aa7-dd12-406f-a8ea-4084e434eb01" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4vtwn" Jul 06 23:57:37.285178 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:37.285156 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mgb8k\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="48453282-83d3-4233-83e5-1943bc2070f7" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k" Jul 06 23:57:37.287122 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:37.287104 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p6drz\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="8e7cec6d-43e8-4668-b785-acbeb9a9aa47" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz" Jul 06 23:57:37.457314 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.457249 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d8fs6"] Jul 06 23:57:37.458873 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:57:37.458842 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee864a87_f13d_418c_9f4e_43dc12ce6add.slice/crio-d0696528de623a498b656cd325749a386ab414578317dfb63c9e38663b926ba6 WatchSource:0}: Error finding container d0696528de623a498b656cd325749a386ab414578317dfb63c9e38663b926ba6: Status 404 returned error can't find the container with id d0696528de623a498b656cd325749a386ab414578317dfb63c9e38663b926ba6 Jul 06 23:57:37.459857 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.459837 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7nh2l"] Jul 06 23:57:37.461781 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:57:37.461756 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92e2d5ca_67d5_46e1_bd03_d236964fe689.slice/crio-2803e16350ee85bae3c3826a0d212844b7311af484156b534a022b48cd5fbe05 WatchSource:0}: Error finding container 2803e16350ee85bae3c3826a0d212844b7311af484156b534a022b48cd5fbe05: Status 404 returned error can't find the container with id 2803e16350ee85bae3c3826a0d212844b7311af484156b534a022b48cd5fbe05 Jul 06 23:57:37.965752 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.965707 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d8fs6" event={"ID":"ee864a87-f13d-418c-9f4e-43dc12ce6add","Type":"ContainerStarted","Data":"4e74a05e2c6a23fa76ebf4444b021ee391e381229cd89412ec7b4d26daa2be71"} Jul 06 23:57:37.966207 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.965809 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d8fs6" event={"ID":"ee864a87-f13d-418c-9f4e-43dc12ce6add","Type":"ContainerStarted","Data":"d0696528de623a498b656cd325749a386ab414578317dfb63c9e38663b926ba6"} Jul 06 23:57:37.966207 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.965840 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d8fs6" Jul 06 23:57:37.967440 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.967417 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wm4s6" event={"ID":"fa56e287-b843-4b0f-b49b-cec8ba10a6de","Type":"ContainerStarted","Data":"225a7add81995092e067309e7d10539d3c49c7da60beaf8bc5a30feb16297c47"} Jul 06 23:57:37.967514 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.967468 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wm4s6" event={"ID":"fa56e287-b843-4b0f-b49b-cec8ba10a6de","Type":"ContainerStarted","Data":"b4769f1195ebca0fe5de849492e54ed156f0dbe0c91146e7d95c776c52ef17bd"} Jul 06 23:57:37.967514 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.967498 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wm4s6" Jul 06 23:57:37.971531 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.971508 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7nh2l" event={"ID":"92e2d5ca-67d5-46e1-bd03-d236964fe689","Type":"ContainerStarted","Data":"27abde383a0d50314aee8a23652b6dd90ba01142e04d6488a551f741c8ded362"} Jul 06 23:57:37.971531 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.971527 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l7fhj" Jul 06 23:57:37.971718 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.971547 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7nh2l" event={"ID":"92e2d5ca-67d5-46e1-bd03-d236964fe689","Type":"ContainerStarted","Data":"2803e16350ee85bae3c3826a0d212844b7311af484156b534a022b48cd5fbe05"} Jul 06 23:57:37.971718 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.971632 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7nh2l" Jul 06 23:57:37.971718 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.971688 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4vtwn" Jul 06 23:57:37.990337 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:37.990307 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p6drz\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="8e7cec6d-43e8-4668-b785-acbeb9a9aa47" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz" Jul 06 23:57:37.990647 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:37.990607 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d8fs6" podStartSLOduration=1.9905934859999999 podStartE2EDuration="1.990593486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:37.988129119 +0000 UTC m=+540.197858665" watchObservedRunningTime="2026-07-06 23:57:37.990593486 +0000 UTC m=+540.200323007" Jul 06 23:57:37.992408 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:37.992387 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-4vtwn\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="657f1aa7-dd12-406f-a8ea-4084e434eb01" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4vtwn" Jul 06 23:57:37.994477 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:37.994456 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mgb8k\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="48453282-83d3-4233-83e5-1943bc2070f7" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k" Jul 06 23:57:37.997043 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:37.997023 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-4vtwn\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="657f1aa7-dd12-406f-a8ea-4084e434eb01" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4vtwn" Jul 06 23:57:38.024920 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:38.024854 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7nh2l" podStartSLOduration=2.024831847 podStartE2EDuration="2.024831847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:38.02342665 +0000 UTC m=+540.233156192" watchObservedRunningTime="2026-07-06 23:57:38.024831847 +0000 UTC m=+540.234561369" Jul 06 23:57:38.056444 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:38.056409 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l7fhj"] Jul 06 23:57:38.065323 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:38.065272 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-l7fhj"] Jul 06 23:57:38.086588 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:38.086535 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wm4s6" podStartSLOduration=2.086518583 podStartE2EDuration="2.086518583s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:38.085501824 +0000 UTC m=+540.295231372" watchObservedRunningTime="2026-07-06 23:57:38.086518583 +0000 UTC m=+540.296248104" Jul 06 23:57:38.087650 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:38.087609 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mgb8k\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="48453282-83d3-4233-83e5-1943bc2070f7" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k" Jul 06 23:57:38.089608 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:38.089576 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p6drz\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="8e7cec6d-43e8-4668-b785-acbeb9a9aa47" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz" Jul 06 23:57:38.303012 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:38.302969 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mgb8k\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="48453282-83d3-4233-83e5-1943bc2070f7" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mgb8k" Jul 06 23:57:38.305032 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:38.305005 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48453282-83d3-4233-83e5-1943bc2070f7" path="/var/lib/kubelet/pods/48453282-83d3-4233-83e5-1943bc2070f7/volumes" Jul 06 23:57:38.305032 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:38.305021 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-p6drz\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="8e7cec6d-43e8-4668-b785-acbeb9a9aa47" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-p6drz" Jul 06 23:57:38.305440 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:38.305426 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f108968-7328-4771-92b3-5db9e1a812dc" path="/var/lib/kubelet/pods/4f108968-7328-4771-92b3-5db9e1a812dc/volumes" Jul 06 23:57:38.305712 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:38.305701 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="657f1aa7-dd12-406f-a8ea-4084e434eb01" path="/var/lib/kubelet/pods/657f1aa7-dd12-406f-a8ea-4084e434eb01/volumes" Jul 06 23:57:38.305924 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:38.305912 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e7cec6d-43e8-4668-b785-acbeb9a9aa47" path="/var/lib/kubelet/pods/8e7cec6d-43e8-4668-b785-acbeb9a9aa47/volumes" Jul 06 23:57:38.306520 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:38.306506 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2c92859-c311-4cf3-a145-8adb24b99b25" path="/var/lib/kubelet/pods/e2c92859-c311-4cf3-a145-8adb24b99b25/volumes" Jul 06 23:57:38.306962 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:38.306943 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-4vtwn\" is forbidden: User \"system:node:ip-10-0-136-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-25.ec2.internal' and this object" podUID="657f1aa7-dd12-406f-a8ea-4084e434eb01" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4vtwn" Jul 06 23:57:48.980130 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:48.980096 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7nh2l" Jul 06 23:57:48.980615 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:48.980193 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-d8fs6" Jul 06 23:57:48.980615 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:48.980243 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wm4s6" Jul 06 23:57:49.038839 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:49.038802 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wm4s6"] Jul 06 23:57:49.039065 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:49.039041 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wm4s6" podUID="fa56e287-b843-4b0f-b49b-cec8ba10a6de" containerName="manager" containerID="cri-o://225a7add81995092e067309e7d10539d3c49c7da60beaf8bc5a30feb16297c47" gracePeriod=10 Jul 06 23:57:49.302691 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:49.302662 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wm4s6" Jul 06 23:57:49.471865 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:49.471830 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cb7zz\" (UniqueName: \"kubernetes.io/projected/fa56e287-b843-4b0f-b49b-cec8ba10a6de-kube-api-access-cb7zz\") pod \"fa56e287-b843-4b0f-b49b-cec8ba10a6de\" (UID: \"fa56e287-b843-4b0f-b49b-cec8ba10a6de\") " Jul 06 23:57:49.471865 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:49.471873 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa56e287-b843-4b0f-b49b-cec8ba10a6de-extensions-socket-volume\") pod \"fa56e287-b843-4b0f-b49b-cec8ba10a6de\" (UID: \"fa56e287-b843-4b0f-b49b-cec8ba10a6de\") " Jul 06 23:57:49.472273 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:49.472243 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa56e287-b843-4b0f-b49b-cec8ba10a6de-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fa56e287-b843-4b0f-b49b-cec8ba10a6de" (UID: "fa56e287-b843-4b0f-b49b-cec8ba10a6de"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:57:49.473953 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:49.473928 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa56e287-b843-4b0f-b49b-cec8ba10a6de-kube-api-access-cb7zz" (OuterVolumeSpecName: "kube-api-access-cb7zz") pod "fa56e287-b843-4b0f-b49b-cec8ba10a6de" (UID: "fa56e287-b843-4b0f-b49b-cec8ba10a6de"). InnerVolumeSpecName "kube-api-access-cb7zz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:57:49.572684 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:49.572586 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cb7zz\" (UniqueName: \"kubernetes.io/projected/fa56e287-b843-4b0f-b49b-cec8ba10a6de-kube-api-access-cb7zz\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:57:49.572684 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:49.572631 2553 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa56e287-b843-4b0f-b49b-cec8ba10a6de-extensions-socket-volume\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:57:50.037863 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:50.037823 2553 generic.go:356] "Generic (PLEG): container finished" podID="fa56e287-b843-4b0f-b49b-cec8ba10a6de" containerID="225a7add81995092e067309e7d10539d3c49c7da60beaf8bc5a30feb16297c47" exitCode=0 Jul 06 23:57:50.037863 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:50.037869 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wm4s6" Jul 06 23:57:50.038449 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:50.037909 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wm4s6" event={"ID":"fa56e287-b843-4b0f-b49b-cec8ba10a6de","Type":"ContainerDied","Data":"225a7add81995092e067309e7d10539d3c49c7da60beaf8bc5a30feb16297c47"} Jul 06 23:57:50.038449 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:50.037969 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wm4s6" event={"ID":"fa56e287-b843-4b0f-b49b-cec8ba10a6de","Type":"ContainerDied","Data":"b4769f1195ebca0fe5de849492e54ed156f0dbe0c91146e7d95c776c52ef17bd"} Jul 06 23:57:50.038449 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:50.037992 2553 scope.go:117] "RemoveContainer" containerID="225a7add81995092e067309e7d10539d3c49c7da60beaf8bc5a30feb16297c47" Jul 06 23:57:50.050332 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:50.050288 2553 scope.go:117] "RemoveContainer" containerID="225a7add81995092e067309e7d10539d3c49c7da60beaf8bc5a30feb16297c47" Jul 06 23:57:50.050696 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:57:50.050673 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"225a7add81995092e067309e7d10539d3c49c7da60beaf8bc5a30feb16297c47\": container with ID starting with 225a7add81995092e067309e7d10539d3c49c7da60beaf8bc5a30feb16297c47 not found: ID does not exist" containerID="225a7add81995092e067309e7d10539d3c49c7da60beaf8bc5a30feb16297c47" Jul 06 23:57:50.050792 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:50.050703 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"225a7add81995092e067309e7d10539d3c49c7da60beaf8bc5a30feb16297c47"} err="failed to get container status \"225a7add81995092e067309e7d10539d3c49c7da60beaf8bc5a30feb16297c47\": rpc error: code = NotFound desc = could not find container \"225a7add81995092e067309e7d10539d3c49c7da60beaf8bc5a30feb16297c47\": container with ID starting with 225a7add81995092e067309e7d10539d3c49c7da60beaf8bc5a30feb16297c47 not found: ID does not exist" Jul 06 23:57:50.073538 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:50.073505 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wm4s6"] Jul 06 23:57:50.083622 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:50.083590 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wm4s6"] Jul 06 23:57:50.307797 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:57:50.307711 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa56e287-b843-4b0f-b49b-cec8ba10a6de" path="/var/lib/kubelet/pods/fa56e287-b843-4b0f-b49b-cec8ba10a6de/volumes" Jul 06 23:58:05.384830 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.384795 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r"] Jul 06 23:58:05.385308 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.385140 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa56e287-b843-4b0f-b49b-cec8ba10a6de" containerName="manager" Jul 06 23:58:05.385308 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.385150 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa56e287-b843-4b0f-b49b-cec8ba10a6de" containerName="manager" Jul 06 23:58:05.385308 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.385223 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa56e287-b843-4b0f-b49b-cec8ba10a6de" containerName="manager" Jul 06 23:58:05.388270 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.388252 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:05.392225 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.392207 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-s2bl5\"" Jul 06 23:58:05.402011 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.401983 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r"] Jul 06 23:58:05.491569 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.491528 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/867fe4ad-91d7-4301-b82a-f5eaf31f863d-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7x84r\" (UID: \"867fe4ad-91d7-4301-b82a-f5eaf31f863d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:05.491788 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.491588 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/867fe4ad-91d7-4301-b82a-f5eaf31f863d-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7x84r\" (UID: \"867fe4ad-91d7-4301-b82a-f5eaf31f863d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:05.491788 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.491619 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/867fe4ad-91d7-4301-b82a-f5eaf31f863d-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7x84r\" (UID: \"867fe4ad-91d7-4301-b82a-f5eaf31f863d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:05.491788 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.491706 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/867fe4ad-91d7-4301-b82a-f5eaf31f863d-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7x84r\" (UID: \"867fe4ad-91d7-4301-b82a-f5eaf31f863d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:05.491788 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.491744 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5jsl\" (UniqueName: \"kubernetes.io/projected/867fe4ad-91d7-4301-b82a-f5eaf31f863d-kube-api-access-d5jsl\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7x84r\" (UID: \"867fe4ad-91d7-4301-b82a-f5eaf31f863d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:05.491788 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.491774 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/867fe4ad-91d7-4301-b82a-f5eaf31f863d-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7x84r\" (UID: \"867fe4ad-91d7-4301-b82a-f5eaf31f863d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:05.492085 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.491807 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/867fe4ad-91d7-4301-b82a-f5eaf31f863d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7x84r\" (UID: \"867fe4ad-91d7-4301-b82a-f5eaf31f863d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:05.492085 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.491834 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/867fe4ad-91d7-4301-b82a-f5eaf31f863d-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7x84r\" (UID: \"867fe4ad-91d7-4301-b82a-f5eaf31f863d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:05.492085 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.491921 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/867fe4ad-91d7-4301-b82a-f5eaf31f863d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7x84r\" (UID: \"867fe4ad-91d7-4301-b82a-f5eaf31f863d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:05.593708 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.593672 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/867fe4ad-91d7-4301-b82a-f5eaf31f863d-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7x84r\" (UID: \"867fe4ad-91d7-4301-b82a-f5eaf31f863d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:05.593708 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.593718 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5jsl\" (UniqueName: \"kubernetes.io/projected/867fe4ad-91d7-4301-b82a-f5eaf31f863d-kube-api-access-d5jsl\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7x84r\" (UID: \"867fe4ad-91d7-4301-b82a-f5eaf31f863d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:05.593945 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.593736 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/867fe4ad-91d7-4301-b82a-f5eaf31f863d-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7x84r\" (UID: \"867fe4ad-91d7-4301-b82a-f5eaf31f863d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:05.593945 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.593861 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/867fe4ad-91d7-4301-b82a-f5eaf31f863d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7x84r\" (UID: \"867fe4ad-91d7-4301-b82a-f5eaf31f863d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:05.593945 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.593903 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/867fe4ad-91d7-4301-b82a-f5eaf31f863d-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7x84r\" (UID: \"867fe4ad-91d7-4301-b82a-f5eaf31f863d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:05.593945 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.593930 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/867fe4ad-91d7-4301-b82a-f5eaf31f863d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7x84r\" (UID: \"867fe4ad-91d7-4301-b82a-f5eaf31f863d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:05.594182 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.594001 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/867fe4ad-91d7-4301-b82a-f5eaf31f863d-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7x84r\" (UID: \"867fe4ad-91d7-4301-b82a-f5eaf31f863d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:05.594182 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.594030 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/867fe4ad-91d7-4301-b82a-f5eaf31f863d-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7x84r\" (UID: \"867fe4ad-91d7-4301-b82a-f5eaf31f863d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:05.594182 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.594122 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/867fe4ad-91d7-4301-b82a-f5eaf31f863d-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7x84r\" (UID: \"867fe4ad-91d7-4301-b82a-f5eaf31f863d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:05.594463 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.594213 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/867fe4ad-91d7-4301-b82a-f5eaf31f863d-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7x84r\" (UID: \"867fe4ad-91d7-4301-b82a-f5eaf31f863d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:05.594523 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.594488 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/867fe4ad-91d7-4301-b82a-f5eaf31f863d-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7x84r\" (UID: \"867fe4ad-91d7-4301-b82a-f5eaf31f863d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:05.594523 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.594511 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/867fe4ad-91d7-4301-b82a-f5eaf31f863d-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7x84r\" (UID: \"867fe4ad-91d7-4301-b82a-f5eaf31f863d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:05.594605 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.594486 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/867fe4ad-91d7-4301-b82a-f5eaf31f863d-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7x84r\" (UID: \"867fe4ad-91d7-4301-b82a-f5eaf31f863d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:05.594713 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.594685 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/867fe4ad-91d7-4301-b82a-f5eaf31f863d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7x84r\" (UID: \"867fe4ad-91d7-4301-b82a-f5eaf31f863d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:05.596589 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.596563 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/867fe4ad-91d7-4301-b82a-f5eaf31f863d-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7x84r\" (UID: \"867fe4ad-91d7-4301-b82a-f5eaf31f863d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:05.596768 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.596750 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/867fe4ad-91d7-4301-b82a-f5eaf31f863d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7x84r\" (UID: \"867fe4ad-91d7-4301-b82a-f5eaf31f863d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:05.609346 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.609317 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5jsl\" (UniqueName: \"kubernetes.io/projected/867fe4ad-91d7-4301-b82a-f5eaf31f863d-kube-api-access-d5jsl\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7x84r\" (UID: \"867fe4ad-91d7-4301-b82a-f5eaf31f863d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:05.610120 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.610102 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/867fe4ad-91d7-4301-b82a-f5eaf31f863d-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-7x84r\" (UID: \"867fe4ad-91d7-4301-b82a-f5eaf31f863d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:05.698193 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.698163 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:05.844554 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.844522 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r"] Jul 06 23:58:05.845759 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:58:05.845736 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod867fe4ad_91d7_4301_b82a_f5eaf31f863d.slice/crio-f3cafbf3d6a3359d374ac7bc1121dfe63dd2f7cccc647538bdc2a2c4ab456333 WatchSource:0}: Error finding container f3cafbf3d6a3359d374ac7bc1121dfe63dd2f7cccc647538bdc2a2c4ab456333: Status 404 returned error can't find the container with id f3cafbf3d6a3359d374ac7bc1121dfe63dd2f7cccc647538bdc2a2c4ab456333 Jul 06 23:58:05.847850 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.847819 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 23:58:05.847939 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.847885 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 23:58:05.847939 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:05.847914 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 23:58:06.128377 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:06.128266 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" event={"ID":"867fe4ad-91d7-4301-b82a-f5eaf31f863d","Type":"ContainerStarted","Data":"f00637479020d544420867a4ea2954febc30213fd1e551367e3635ebcbc958f6"} Jul 06 23:58:06.128377 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:06.128343 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" event={"ID":"867fe4ad-91d7-4301-b82a-f5eaf31f863d","Type":"ContainerStarted","Data":"f3cafbf3d6a3359d374ac7bc1121dfe63dd2f7cccc647538bdc2a2c4ab456333"} Jul 06 23:58:06.147902 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:06.147843 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" podStartSLOduration=1.147823965 podStartE2EDuration="1.147823965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:58:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:06.147607707 +0000 UTC m=+568.357337230" watchObservedRunningTime="2026-07-06 23:58:06.147823965 +0000 UTC m=+568.357553486" Jul 06 23:58:06.699237 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:06.699195 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:06.704211 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:06.704184 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:07.128164 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:07.128056 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6szxw"] Jul 06 23:58:07.131801 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:07.131776 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6szxw" Jul 06 23:58:07.133947 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:07.133918 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:07.134823 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:07.134800 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-fhz99\"" Jul 06 23:58:07.134950 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:07.134866 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 06 23:58:07.135051 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:07.135037 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:58:07.141395 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:07.141366 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6szxw"] Jul 06 23:58:07.210465 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:07.210423 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cjf7\" (UniqueName: \"kubernetes.io/projected/00b3dee3-1f88-44d9-abc3-1fd8382ab7b6-kube-api-access-2cjf7\") pod \"limitador-limitador-64c94fd64f-6szxw\" (UID: \"00b3dee3-1f88-44d9-abc3-1fd8382ab7b6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6szxw" Jul 06 23:58:07.210680 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:07.210547 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/00b3dee3-1f88-44d9-abc3-1fd8382ab7b6-config-file\") pod \"limitador-limitador-64c94fd64f-6szxw\" (UID: \"00b3dee3-1f88-44d9-abc3-1fd8382ab7b6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6szxw" Jul 06 23:58:07.225980 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:07.225940 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6szxw"] Jul 06 23:58:07.311383 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:07.311350 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/00b3dee3-1f88-44d9-abc3-1fd8382ab7b6-config-file\") pod \"limitador-limitador-64c94fd64f-6szxw\" (UID: \"00b3dee3-1f88-44d9-abc3-1fd8382ab7b6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6szxw" Jul 06 23:58:07.311564 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:07.311508 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cjf7\" (UniqueName: \"kubernetes.io/projected/00b3dee3-1f88-44d9-abc3-1fd8382ab7b6-kube-api-access-2cjf7\") pod \"limitador-limitador-64c94fd64f-6szxw\" (UID: \"00b3dee3-1f88-44d9-abc3-1fd8382ab7b6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6szxw" Jul 06 23:58:07.312044 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:07.312021 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/00b3dee3-1f88-44d9-abc3-1fd8382ab7b6-config-file\") pod \"limitador-limitador-64c94fd64f-6szxw\" (UID: \"00b3dee3-1f88-44d9-abc3-1fd8382ab7b6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6szxw" Jul 06 23:58:07.321944 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:07.321914 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cjf7\" (UniqueName: \"kubernetes.io/projected/00b3dee3-1f88-44d9-abc3-1fd8382ab7b6-kube-api-access-2cjf7\") pod \"limitador-limitador-64c94fd64f-6szxw\" (UID: \"00b3dee3-1f88-44d9-abc3-1fd8382ab7b6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6szxw" Jul 06 23:58:07.445579 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:07.445538 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6szxw" Jul 06 23:58:07.595831 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:07.595797 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6szxw"] Jul 06 23:58:07.597758 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:58:07.597731 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00b3dee3_1f88_44d9_abc3_1fd8382ab7b6.slice/crio-46b7b72ac58de2465e6987ed77e01b2526449e70db5aef858c04b22ec2e88b3e WatchSource:0}: Error finding container 46b7b72ac58de2465e6987ed77e01b2526449e70db5aef858c04b22ec2e88b3e: Status 404 returned error can't find the container with id 46b7b72ac58de2465e6987ed77e01b2526449e70db5aef858c04b22ec2e88b3e Jul 06 23:58:08.140814 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:08.140777 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6szxw" event={"ID":"00b3dee3-1f88-44d9-abc3-1fd8382ab7b6","Type":"ContainerStarted","Data":"46b7b72ac58de2465e6987ed77e01b2526449e70db5aef858c04b22ec2e88b3e"} Jul 06 23:58:11.160687 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:11.160648 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6szxw" event={"ID":"00b3dee3-1f88-44d9-abc3-1fd8382ab7b6","Type":"ContainerStarted","Data":"b7f1302e2f8be482875d395af517b6c7643f804ac7ed95e4f0f641a43541bcd9"} Jul 06 23:58:11.161197 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:11.160770 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-6szxw" Jul 06 23:58:11.178159 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:11.178096 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-6szxw" podStartSLOduration=1.478541285 podStartE2EDuration="4.178073983s" totalImagesPullingTime="2.699532698s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:58:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:11.176668227 +0000 UTC m=+573.386397749" watchObservedRunningTime="2026-07-06 23:58:11.178073983 +0000 UTC m=+573.387803505" Jul 06 23:58:22.134541 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:22.134502 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6szxw"] Jul 06 23:58:22.135066 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:22.134761 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-6szxw" podUID="00b3dee3-1f88-44d9-abc3-1fd8382ab7b6" containerName="limitador" containerID="cri-o://b7f1302e2f8be482875d395af517b6c7643f804ac7ed95e4f0f641a43541bcd9" gracePeriod=30 Jul 06 23:58:22.135727 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:22.135418 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-6szxw" Jul 06 23:58:22.687085 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:22.687059 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6szxw" Jul 06 23:58:22.763701 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:22.763607 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cjf7\" (UniqueName: \"kubernetes.io/projected/00b3dee3-1f88-44d9-abc3-1fd8382ab7b6-kube-api-access-2cjf7\") pod \"00b3dee3-1f88-44d9-abc3-1fd8382ab7b6\" (UID: \"00b3dee3-1f88-44d9-abc3-1fd8382ab7b6\") " Jul 06 23:58:22.763701 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:22.763677 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/00b3dee3-1f88-44d9-abc3-1fd8382ab7b6-config-file\") pod \"00b3dee3-1f88-44d9-abc3-1fd8382ab7b6\" (UID: \"00b3dee3-1f88-44d9-abc3-1fd8382ab7b6\") " Jul 06 23:58:22.764095 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:22.764048 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00b3dee3-1f88-44d9-abc3-1fd8382ab7b6-config-file" (OuterVolumeSpecName: "config-file") pod "00b3dee3-1f88-44d9-abc3-1fd8382ab7b6" (UID: "00b3dee3-1f88-44d9-abc3-1fd8382ab7b6"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:58:22.765759 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:22.765737 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00b3dee3-1f88-44d9-abc3-1fd8382ab7b6-kube-api-access-2cjf7" (OuterVolumeSpecName: "kube-api-access-2cjf7") pod "00b3dee3-1f88-44d9-abc3-1fd8382ab7b6" (UID: "00b3dee3-1f88-44d9-abc3-1fd8382ab7b6"). InnerVolumeSpecName "kube-api-access-2cjf7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:58:22.864535 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:22.864501 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cjf7\" (UniqueName: \"kubernetes.io/projected/00b3dee3-1f88-44d9-abc3-1fd8382ab7b6-kube-api-access-2cjf7\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:58:22.864535 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:22.864531 2553 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/00b3dee3-1f88-44d9-abc3-1fd8382ab7b6-config-file\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:58:23.234925 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:23.234889 2553 generic.go:356] "Generic (PLEG): container finished" podID="00b3dee3-1f88-44d9-abc3-1fd8382ab7b6" containerID="b7f1302e2f8be482875d395af517b6c7643f804ac7ed95e4f0f641a43541bcd9" exitCode=0 Jul 06 23:58:23.235390 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:23.234932 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6szxw" Jul 06 23:58:23.235390 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:23.234976 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6szxw" event={"ID":"00b3dee3-1f88-44d9-abc3-1fd8382ab7b6","Type":"ContainerDied","Data":"b7f1302e2f8be482875d395af517b6c7643f804ac7ed95e4f0f641a43541bcd9"} Jul 06 23:58:23.235390 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:23.235028 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6szxw" event={"ID":"00b3dee3-1f88-44d9-abc3-1fd8382ab7b6","Type":"ContainerDied","Data":"46b7b72ac58de2465e6987ed77e01b2526449e70db5aef858c04b22ec2e88b3e"} Jul 06 23:58:23.235390 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:23.235057 2553 scope.go:117] "RemoveContainer" containerID="b7f1302e2f8be482875d395af517b6c7643f804ac7ed95e4f0f641a43541bcd9" Jul 06 23:58:23.244720 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:23.244704 2553 scope.go:117] "RemoveContainer" containerID="b7f1302e2f8be482875d395af517b6c7643f804ac7ed95e4f0f641a43541bcd9" Jul 06 23:58:23.245075 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:58:23.245046 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7f1302e2f8be482875d395af517b6c7643f804ac7ed95e4f0f641a43541bcd9\": container with ID starting with b7f1302e2f8be482875d395af517b6c7643f804ac7ed95e4f0f641a43541bcd9 not found: ID does not exist" containerID="b7f1302e2f8be482875d395af517b6c7643f804ac7ed95e4f0f641a43541bcd9" Jul 06 23:58:23.245162 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:23.245084 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7f1302e2f8be482875d395af517b6c7643f804ac7ed95e4f0f641a43541bcd9"} err="failed to get container status \"b7f1302e2f8be482875d395af517b6c7643f804ac7ed95e4f0f641a43541bcd9\": rpc error: code = NotFound desc = could not find container \"b7f1302e2f8be482875d395af517b6c7643f804ac7ed95e4f0f641a43541bcd9\": container with ID starting with b7f1302e2f8be482875d395af517b6c7643f804ac7ed95e4f0f641a43541bcd9 not found: ID does not exist" Jul 06 23:58:23.257182 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:23.257151 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6szxw"] Jul 06 23:58:23.260135 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:23.260106 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6szxw"] Jul 06 23:58:24.303048 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:24.303017 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00b3dee3-1f88-44d9-abc3-1fd8382ab7b6" path="/var/lib/kubelet/pods/00b3dee3-1f88-44d9-abc3-1fd8382ab7b6/volumes" Jul 06 23:58:28.711779 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:28.711736 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-mjkm4"] Jul 06 23:58:28.712189 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:28.712169 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00b3dee3-1f88-44d9-abc3-1fd8382ab7b6" containerName="limitador" Jul 06 23:58:28.712236 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:28.712193 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="00b3dee3-1f88-44d9-abc3-1fd8382ab7b6" containerName="limitador" Jul 06 23:58:28.712311 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:28.712286 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="00b3dee3-1f88-44d9-abc3-1fd8382ab7b6" containerName="limitador" Jul 06 23:58:28.716739 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:28.716712 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-mjkm4" Jul 06 23:58:28.719699 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:28.719569 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 06 23:58:28.719952 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:28.719926 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-rrxhj\"" Jul 06 23:58:28.722677 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:28.722204 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-mjkm4"] Jul 06 23:58:28.818176 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:28.818135 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7gcw\" (UniqueName: \"kubernetes.io/projected/d27d869f-2a23-476d-9006-3e5c71823782-kube-api-access-b7gcw\") pod \"postgres-69fb4594cf-mjkm4\" (UID: \"d27d869f-2a23-476d-9006-3e5c71823782\") " pod="opendatahub/postgres-69fb4594cf-mjkm4" Jul 06 23:58:28.818384 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:28.818289 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d27d869f-2a23-476d-9006-3e5c71823782-data\") pod \"postgres-69fb4594cf-mjkm4\" (UID: \"d27d869f-2a23-476d-9006-3e5c71823782\") " pod="opendatahub/postgres-69fb4594cf-mjkm4" Jul 06 23:58:28.918960 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:28.918917 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d27d869f-2a23-476d-9006-3e5c71823782-data\") pod \"postgres-69fb4594cf-mjkm4\" (UID: \"d27d869f-2a23-476d-9006-3e5c71823782\") " pod="opendatahub/postgres-69fb4594cf-mjkm4" Jul 06 23:58:28.919186 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:28.918988 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7gcw\" (UniqueName: \"kubernetes.io/projected/d27d869f-2a23-476d-9006-3e5c71823782-kube-api-access-b7gcw\") pod \"postgres-69fb4594cf-mjkm4\" (UID: \"d27d869f-2a23-476d-9006-3e5c71823782\") " pod="opendatahub/postgres-69fb4594cf-mjkm4" Jul 06 23:58:28.919421 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:28.919401 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d27d869f-2a23-476d-9006-3e5c71823782-data\") pod \"postgres-69fb4594cf-mjkm4\" (UID: \"d27d869f-2a23-476d-9006-3e5c71823782\") " pod="opendatahub/postgres-69fb4594cf-mjkm4" Jul 06 23:58:28.927754 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:28.927728 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7gcw\" (UniqueName: \"kubernetes.io/projected/d27d869f-2a23-476d-9006-3e5c71823782-kube-api-access-b7gcw\") pod \"postgres-69fb4594cf-mjkm4\" (UID: \"d27d869f-2a23-476d-9006-3e5c71823782\") " pod="opendatahub/postgres-69fb4594cf-mjkm4" Jul 06 23:58:29.029726 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:29.029630 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-mjkm4" Jul 06 23:58:29.169734 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:29.169706 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-mjkm4"] Jul 06 23:58:29.171617 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:58:29.171588 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd27d869f_2a23_476d_9006_3e5c71823782.slice/crio-0270c7b00a81460d30d286e730fbc0cae248aad421c7032739380b98f215e1fa WatchSource:0}: Error finding container 0270c7b00a81460d30d286e730fbc0cae248aad421c7032739380b98f215e1fa: Status 404 returned error can't find the container with id 0270c7b00a81460d30d286e730fbc0cae248aad421c7032739380b98f215e1fa Jul 06 23:58:29.270389 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:29.270352 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-mjkm4" event={"ID":"d27d869f-2a23-476d-9006-3e5c71823782","Type":"ContainerStarted","Data":"0270c7b00a81460d30d286e730fbc0cae248aad421c7032739380b98f215e1fa"} Jul 06 23:58:34.310180 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:34.310074 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-mjkm4" event={"ID":"d27d869f-2a23-476d-9006-3e5c71823782","Type":"ContainerStarted","Data":"fcc304bdc5b296701730c3780c3bb43d88025041ea5f9201ef32e1ab74ec95e4"} Jul 06 23:58:34.310180 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:34.310153 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-mjkm4" Jul 06 23:58:34.327631 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:34.327577 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-mjkm4" podStartSLOduration=1.566862615 podStartE2EDuration="6.327557286s" totalImagesPullingTime="4.760694671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:58:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:34.324962406 +0000 UTC m=+596.534691944" watchObservedRunningTime="2026-07-06 23:58:34.327557286 +0000 UTC m=+596.537286807" Jul 06 23:58:38.187312 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:38.187269 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4z8vq_3253bdfe-5248-4d4c-9864-11660eb042ca/ovn-acl-logging/0.log" Jul 06 23:58:38.194583 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:38.194548 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4z8vq_3253bdfe-5248-4d4c-9864-11660eb042ca/ovn-acl-logging/0.log" Jul 06 23:58:40.346785 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:40.346745 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-mjkm4" Jul 06 23:58:47.023481 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:47.023439 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-869747c694-4fkpb"] Jul 06 23:58:47.028209 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:47.028189 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-869747c694-4fkpb" Jul 06 23:58:47.031213 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:47.031179 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-rlhkj\"" Jul 06 23:58:47.031376 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:47.031182 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 23:58:47.032339 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:47.032322 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 23:58:47.039426 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:47.039399 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-869747c694-4fkpb"] Jul 06 23:58:47.094903 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:47.094868 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8s4t\" (UniqueName: \"kubernetes.io/projected/abf2bf9d-bd82-469f-8981-3a93b24199a0-kube-api-access-b8s4t\") pod \"maas-controller-869747c694-4fkpb\" (UID: \"abf2bf9d-bd82-469f-8981-3a93b24199a0\") " pod="opendatahub/maas-controller-869747c694-4fkpb" Jul 06 23:58:47.095085 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:47.094938 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/abf2bf9d-bd82-469f-8981-3a93b24199a0-webhook-cert\") pod \"maas-controller-869747c694-4fkpb\" (UID: \"abf2bf9d-bd82-469f-8981-3a93b24199a0\") " pod="opendatahub/maas-controller-869747c694-4fkpb" Jul 06 23:58:47.196305 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:47.196252 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/abf2bf9d-bd82-469f-8981-3a93b24199a0-webhook-cert\") pod \"maas-controller-869747c694-4fkpb\" (UID: \"abf2bf9d-bd82-469f-8981-3a93b24199a0\") " pod="opendatahub/maas-controller-869747c694-4fkpb" Jul 06 23:58:47.196487 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:47.196378 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8s4t\" (UniqueName: \"kubernetes.io/projected/abf2bf9d-bd82-469f-8981-3a93b24199a0-kube-api-access-b8s4t\") pod \"maas-controller-869747c694-4fkpb\" (UID: \"abf2bf9d-bd82-469f-8981-3a93b24199a0\") " pod="opendatahub/maas-controller-869747c694-4fkpb" Jul 06 23:58:47.196487 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:58:47.196417 2553 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 06 23:58:47.196560 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:58:47.196494 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abf2bf9d-bd82-469f-8981-3a93b24199a0-webhook-cert podName:abf2bf9d-bd82-469f-8981-3a93b24199a0 nodeName:}" failed. No retries permitted until 2026-07-06 23:58:47.696474848 +0000 UTC m=+609.906204357 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/abf2bf9d-bd82-469f-8981-3a93b24199a0-webhook-cert") pod "maas-controller-869747c694-4fkpb" (UID: "abf2bf9d-bd82-469f-8981-3a93b24199a0") : secret "maas-controller-webhook-cert" not found Jul 06 23:58:47.203668 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:47.203637 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-56bf7f957f-llwjm"] Jul 06 23:58:47.208213 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:47.208191 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56bf7f957f-llwjm" Jul 06 23:58:47.209437 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:47.209411 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8s4t\" (UniqueName: \"kubernetes.io/projected/abf2bf9d-bd82-469f-8981-3a93b24199a0-kube-api-access-b8s4t\") pod \"maas-controller-869747c694-4fkpb\" (UID: \"abf2bf9d-bd82-469f-8981-3a93b24199a0\") " pod="opendatahub/maas-controller-869747c694-4fkpb" Jul 06 23:58:47.220594 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:47.220564 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56bf7f957f-llwjm"] Jul 06 23:58:47.297393 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:47.297287 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgn2q\" (UniqueName: \"kubernetes.io/projected/1c66b153-6263-4f23-998b-e8f4ef77a9dd-kube-api-access-fgn2q\") pod \"maas-controller-56bf7f957f-llwjm\" (UID: \"1c66b153-6263-4f23-998b-e8f4ef77a9dd\") " pod="opendatahub/maas-controller-56bf7f957f-llwjm" Jul 06 23:58:47.297572 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:47.297398 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1c66b153-6263-4f23-998b-e8f4ef77a9dd-webhook-cert\") pod \"maas-controller-56bf7f957f-llwjm\" (UID: \"1c66b153-6263-4f23-998b-e8f4ef77a9dd\") " pod="opendatahub/maas-controller-56bf7f957f-llwjm" Jul 06 23:58:47.398319 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:47.398254 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1c66b153-6263-4f23-998b-e8f4ef77a9dd-webhook-cert\") pod \"maas-controller-56bf7f957f-llwjm\" (UID: \"1c66b153-6263-4f23-998b-e8f4ef77a9dd\") " pod="opendatahub/maas-controller-56bf7f957f-llwjm" Jul 06 23:58:47.398508 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:47.398356 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgn2q\" (UniqueName: \"kubernetes.io/projected/1c66b153-6263-4f23-998b-e8f4ef77a9dd-kube-api-access-fgn2q\") pod \"maas-controller-56bf7f957f-llwjm\" (UID: \"1c66b153-6263-4f23-998b-e8f4ef77a9dd\") " pod="opendatahub/maas-controller-56bf7f957f-llwjm" Jul 06 23:58:47.398508 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:58:47.398411 2553 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 06 23:58:47.398508 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:58:47.398474 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c66b153-6263-4f23-998b-e8f4ef77a9dd-webhook-cert podName:1c66b153-6263-4f23-998b-e8f4ef77a9dd nodeName:}" failed. No retries permitted until 2026-07-06 23:58:47.898459725 +0000 UTC m=+610.108189230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/1c66b153-6263-4f23-998b-e8f4ef77a9dd-webhook-cert") pod "maas-controller-56bf7f957f-llwjm" (UID: "1c66b153-6263-4f23-998b-e8f4ef77a9dd") : secret "maas-controller-webhook-cert" not found Jul 06 23:58:47.407671 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:47.407634 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgn2q\" (UniqueName: \"kubernetes.io/projected/1c66b153-6263-4f23-998b-e8f4ef77a9dd-kube-api-access-fgn2q\") pod \"maas-controller-56bf7f957f-llwjm\" (UID: \"1c66b153-6263-4f23-998b-e8f4ef77a9dd\") " pod="opendatahub/maas-controller-56bf7f957f-llwjm" Jul 06 23:58:47.701169 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:47.701127 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/abf2bf9d-bd82-469f-8981-3a93b24199a0-webhook-cert\") pod \"maas-controller-869747c694-4fkpb\" (UID: \"abf2bf9d-bd82-469f-8981-3a93b24199a0\") " pod="opendatahub/maas-controller-869747c694-4fkpb" Jul 06 23:58:47.703636 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:47.703603 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/abf2bf9d-bd82-469f-8981-3a93b24199a0-webhook-cert\") pod \"maas-controller-869747c694-4fkpb\" (UID: \"abf2bf9d-bd82-469f-8981-3a93b24199a0\") " pod="opendatahub/maas-controller-869747c694-4fkpb" Jul 06 23:58:47.903327 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:47.903258 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1c66b153-6263-4f23-998b-e8f4ef77a9dd-webhook-cert\") pod \"maas-controller-56bf7f957f-llwjm\" (UID: \"1c66b153-6263-4f23-998b-e8f4ef77a9dd\") " pod="opendatahub/maas-controller-56bf7f957f-llwjm" Jul 06 23:58:47.905774 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:47.905744 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1c66b153-6263-4f23-998b-e8f4ef77a9dd-webhook-cert\") pod \"maas-controller-56bf7f957f-llwjm\" (UID: \"1c66b153-6263-4f23-998b-e8f4ef77a9dd\") " pod="opendatahub/maas-controller-56bf7f957f-llwjm" Jul 06 23:58:47.938819 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:47.938783 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-869747c694-4fkpb" Jul 06 23:58:48.079432 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:48.079406 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-869747c694-4fkpb"] Jul 06 23:58:48.081453 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:58:48.081427 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabf2bf9d_bd82_469f_8981_3a93b24199a0.slice/crio-f07e670093cfed0607a656a7e9539f7d3989047f8fb33cfd8bf91ac67bd8f4f0 WatchSource:0}: Error finding container f07e670093cfed0607a656a7e9539f7d3989047f8fb33cfd8bf91ac67bd8f4f0: Status 404 returned error can't find the container with id f07e670093cfed0607a656a7e9539f7d3989047f8fb33cfd8bf91ac67bd8f4f0 Jul 06 23:58:48.127452 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:48.127416 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56bf7f957f-llwjm" Jul 06 23:58:48.269914 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:48.269889 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56bf7f957f-llwjm"] Jul 06 23:58:48.271823 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:58:48.271796 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c66b153_6263_4f23_998b_e8f4ef77a9dd.slice/crio-39a148540bbe79d191ca9b1ce20dd315e728880a7b4234826007aa5d2757d04d WatchSource:0}: Error finding container 39a148540bbe79d191ca9b1ce20dd315e728880a7b4234826007aa5d2757d04d: Status 404 returned error can't find the container with id 39a148540bbe79d191ca9b1ce20dd315e728880a7b4234826007aa5d2757d04d Jul 06 23:58:48.385715 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:48.385681 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56bf7f957f-llwjm" event={"ID":"1c66b153-6263-4f23-998b-e8f4ef77a9dd","Type":"ContainerStarted","Data":"39a148540bbe79d191ca9b1ce20dd315e728880a7b4234826007aa5d2757d04d"} Jul 06 23:58:48.386902 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:48.386874 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-869747c694-4fkpb" event={"ID":"abf2bf9d-bd82-469f-8981-3a93b24199a0","Type":"ContainerStarted","Data":"f07e670093cfed0607a656a7e9539f7d3989047f8fb33cfd8bf91ac67bd8f4f0"} Jul 06 23:58:50.401225 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:50.401186 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56bf7f957f-llwjm" event={"ID":"1c66b153-6263-4f23-998b-e8f4ef77a9dd","Type":"ContainerStarted","Data":"dcdca9451e40b58f87156894b45eff85f0590d9377596150c9ddf31da7ec0e9b"} Jul 06 23:58:50.401810 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:50.401285 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-56bf7f957f-llwjm" Jul 06 23:58:50.403646 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:50.403617 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-869747c694-4fkpb" event={"ID":"abf2bf9d-bd82-469f-8981-3a93b24199a0","Type":"ContainerStarted","Data":"8f8c78738f965b9a2344b05770213470fd4cac4f82d53ef77e6cd09aafaeb2bb"} Jul 06 23:58:50.403784 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:50.403766 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-869747c694-4fkpb" Jul 06 23:58:50.424902 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:50.424851 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-56bf7f957f-llwjm" podStartSLOduration=1.5803323169999999 podStartE2EDuration="3.424832236s" totalImagesPullingTime="1.844499919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:58:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:50.421831625 +0000 UTC m=+612.631561149" watchObservedRunningTime="2026-07-06 23:58:50.424832236 +0000 UTC m=+612.634561757" Jul 06 23:58:50.446612 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:58:50.446550 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-869747c694-4fkpb" podStartSLOduration=2.399456069 podStartE2EDuration="4.446527253s" totalImagesPullingTime="2.047071184s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:58:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:50.445630624 +0000 UTC m=+612.655360146" watchObservedRunningTime="2026-07-06 23:58:50.446527253 +0000 UTC m=+612.656256775" Jul 06 23:59:01.413947 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:01.413867 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-869747c694-4fkpb" Jul 06 23:59:01.414479 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:01.414161 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-56bf7f957f-llwjm" Jul 06 23:59:01.472544 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:01.472506 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-869747c694-4fkpb"] Jul 06 23:59:01.472741 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:01.472716 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-869747c694-4fkpb" podUID="abf2bf9d-bd82-469f-8981-3a93b24199a0" containerName="manager" containerID="cri-o://8f8c78738f965b9a2344b05770213470fd4cac4f82d53ef77e6cd09aafaeb2bb" gracePeriod=30 Jul 06 23:59:01.727462 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:01.727436 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-869747c694-4fkpb" Jul 06 23:59:01.827746 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:01.827677 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8s4t\" (UniqueName: \"kubernetes.io/projected/abf2bf9d-bd82-469f-8981-3a93b24199a0-kube-api-access-b8s4t\") pod \"abf2bf9d-bd82-469f-8981-3a93b24199a0\" (UID: \"abf2bf9d-bd82-469f-8981-3a93b24199a0\") " Jul 06 23:59:01.827746 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:01.827758 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/abf2bf9d-bd82-469f-8981-3a93b24199a0-webhook-cert\") pod \"abf2bf9d-bd82-469f-8981-3a93b24199a0\" (UID: \"abf2bf9d-bd82-469f-8981-3a93b24199a0\") " Jul 06 23:59:01.830467 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:01.830409 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abf2bf9d-bd82-469f-8981-3a93b24199a0-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "abf2bf9d-bd82-469f-8981-3a93b24199a0" (UID: "abf2bf9d-bd82-469f-8981-3a93b24199a0"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:59:01.830467 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:01.830447 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abf2bf9d-bd82-469f-8981-3a93b24199a0-kube-api-access-b8s4t" (OuterVolumeSpecName: "kube-api-access-b8s4t") pod "abf2bf9d-bd82-469f-8981-3a93b24199a0" (UID: "abf2bf9d-bd82-469f-8981-3a93b24199a0"). InnerVolumeSpecName "kube-api-access-b8s4t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:59:01.929272 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:01.929230 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8s4t\" (UniqueName: \"kubernetes.io/projected/abf2bf9d-bd82-469f-8981-3a93b24199a0-kube-api-access-b8s4t\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:59:01.929272 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:01.929265 2553 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/abf2bf9d-bd82-469f-8981-3a93b24199a0-webhook-cert\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 06 23:59:02.467927 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:02.467889 2553 generic.go:356] "Generic (PLEG): container finished" podID="abf2bf9d-bd82-469f-8981-3a93b24199a0" containerID="8f8c78738f965b9a2344b05770213470fd4cac4f82d53ef77e6cd09aafaeb2bb" exitCode=0 Jul 06 23:59:02.468411 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:02.467939 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-869747c694-4fkpb" event={"ID":"abf2bf9d-bd82-469f-8981-3a93b24199a0","Type":"ContainerDied","Data":"8f8c78738f965b9a2344b05770213470fd4cac4f82d53ef77e6cd09aafaeb2bb"} Jul 06 23:59:02.468411 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:02.467956 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-869747c694-4fkpb" Jul 06 23:59:02.468411 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:02.467992 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-869747c694-4fkpb" event={"ID":"abf2bf9d-bd82-469f-8981-3a93b24199a0","Type":"ContainerDied","Data":"f07e670093cfed0607a656a7e9539f7d3989047f8fb33cfd8bf91ac67bd8f4f0"} Jul 06 23:59:02.468411 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:02.468015 2553 scope.go:117] "RemoveContainer" containerID="8f8c78738f965b9a2344b05770213470fd4cac4f82d53ef77e6cd09aafaeb2bb" Jul 06 23:59:02.477545 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:02.477242 2553 scope.go:117] "RemoveContainer" containerID="8f8c78738f965b9a2344b05770213470fd4cac4f82d53ef77e6cd09aafaeb2bb" Jul 06 23:59:02.477614 ip-10-0-136-25 kubenswrapper[2553]: E0706 23:59:02.477592 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f8c78738f965b9a2344b05770213470fd4cac4f82d53ef77e6cd09aafaeb2bb\": container with ID starting with 8f8c78738f965b9a2344b05770213470fd4cac4f82d53ef77e6cd09aafaeb2bb not found: ID does not exist" containerID="8f8c78738f965b9a2344b05770213470fd4cac4f82d53ef77e6cd09aafaeb2bb" Jul 06 23:59:02.477655 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:02.477621 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f8c78738f965b9a2344b05770213470fd4cac4f82d53ef77e6cd09aafaeb2bb"} err="failed to get container status \"8f8c78738f965b9a2344b05770213470fd4cac4f82d53ef77e6cd09aafaeb2bb\": rpc error: code = NotFound desc = could not find container \"8f8c78738f965b9a2344b05770213470fd4cac4f82d53ef77e6cd09aafaeb2bb\": container with ID starting with 8f8c78738f965b9a2344b05770213470fd4cac4f82d53ef77e6cd09aafaeb2bb not found: ID does not exist" Jul 06 23:59:02.485737 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:02.485707 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-869747c694-4fkpb"] Jul 06 23:59:02.489594 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:02.489564 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-869747c694-4fkpb"] Jul 06 23:59:04.302852 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:04.302816 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abf2bf9d-bd82-469f-8981-3a93b24199a0" path="/var/lib/kubelet/pods/abf2bf9d-bd82-469f-8981-3a93b24199a0/volumes" Jul 06 23:59:21.669012 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:21.668972 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-7bdf95f944-bdxls"] Jul 06 23:59:21.669520 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:21.669501 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abf2bf9d-bd82-469f-8981-3a93b24199a0" containerName="manager" Jul 06 23:59:21.669567 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:21.669522 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="abf2bf9d-bd82-469f-8981-3a93b24199a0" containerName="manager" Jul 06 23:59:21.669606 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:21.669581 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="abf2bf9d-bd82-469f-8981-3a93b24199a0" containerName="manager" Jul 06 23:59:21.691947 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:21.691918 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-7bdf95f944-bdxls"] Jul 06 23:59:21.692115 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:21.692022 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-7bdf95f944-bdxls" Jul 06 23:59:21.695205 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:21.695182 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 06 23:59:21.695376 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:21.695345 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-8lgjp\"" Jul 06 23:59:21.818995 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:21.818965 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/818a2117-421b-478b-9633-a95a0a927158-maas-api-tls\") pod \"maas-api-7bdf95f944-bdxls\" (UID: \"818a2117-421b-478b-9633-a95a0a927158\") " pod="opendatahub/maas-api-7bdf95f944-bdxls" Jul 06 23:59:21.819188 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:21.819006 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mldxm\" (UniqueName: \"kubernetes.io/projected/818a2117-421b-478b-9633-a95a0a927158-kube-api-access-mldxm\") pod \"maas-api-7bdf95f944-bdxls\" (UID: \"818a2117-421b-478b-9633-a95a0a927158\") " pod="opendatahub/maas-api-7bdf95f944-bdxls" Jul 06 23:59:21.819188 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:21.819103 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/818a2117-421b-478b-9633-a95a0a927158-openshift-service-ca\") pod \"maas-api-7bdf95f944-bdxls\" (UID: \"818a2117-421b-478b-9633-a95a0a927158\") " pod="opendatahub/maas-api-7bdf95f944-bdxls" Jul 06 23:59:21.920610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:21.920520 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/818a2117-421b-478b-9633-a95a0a927158-openshift-service-ca\") pod \"maas-api-7bdf95f944-bdxls\" (UID: \"818a2117-421b-478b-9633-a95a0a927158\") " pod="opendatahub/maas-api-7bdf95f944-bdxls" Jul 06 23:59:21.920610 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:21.920601 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/818a2117-421b-478b-9633-a95a0a927158-maas-api-tls\") pod \"maas-api-7bdf95f944-bdxls\" (UID: \"818a2117-421b-478b-9633-a95a0a927158\") " pod="opendatahub/maas-api-7bdf95f944-bdxls" Jul 06 23:59:21.920848 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:21.920642 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mldxm\" (UniqueName: \"kubernetes.io/projected/818a2117-421b-478b-9633-a95a0a927158-kube-api-access-mldxm\") pod \"maas-api-7bdf95f944-bdxls\" (UID: \"818a2117-421b-478b-9633-a95a0a927158\") " pod="opendatahub/maas-api-7bdf95f944-bdxls" Jul 06 23:59:21.921430 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:21.921408 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/818a2117-421b-478b-9633-a95a0a927158-openshift-service-ca\") pod \"maas-api-7bdf95f944-bdxls\" (UID: \"818a2117-421b-478b-9633-a95a0a927158\") " pod="opendatahub/maas-api-7bdf95f944-bdxls" Jul 06 23:59:21.923152 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:21.923129 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/818a2117-421b-478b-9633-a95a0a927158-maas-api-tls\") pod \"maas-api-7bdf95f944-bdxls\" (UID: \"818a2117-421b-478b-9633-a95a0a927158\") " pod="opendatahub/maas-api-7bdf95f944-bdxls" Jul 06 23:59:21.928908 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:21.928884 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mldxm\" (UniqueName: \"kubernetes.io/projected/818a2117-421b-478b-9633-a95a0a927158-kube-api-access-mldxm\") pod \"maas-api-7bdf95f944-bdxls\" (UID: \"818a2117-421b-478b-9633-a95a0a927158\") " pod="opendatahub/maas-api-7bdf95f944-bdxls" Jul 06 23:59:22.003598 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:22.003466 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-7bdf95f944-bdxls" Jul 06 23:59:22.149913 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:22.149877 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-tmjqm"] Jul 06 23:59:22.155052 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:22.155027 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-tmjqm" Jul 06 23:59:22.158013 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:22.157989 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-dtf66\"" Jul 06 23:59:22.158165 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:22.158017 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 06 23:59:22.170705 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:22.170640 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-tmjqm"] Jul 06 23:59:22.183887 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:22.183848 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-2bhlm"] Jul 06 23:59:22.188214 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:22.188189 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-2bhlm" Jul 06 23:59:22.191344 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:22.190143 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-7bdf95f944-bdxls"] Jul 06 23:59:22.192586 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:22.192564 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 23:59:22.198023 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:22.197997 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-2bhlm"] Jul 06 23:59:22.224833 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:22.224793 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6582771-54aa-4c55-b416-3f661db37add-plugins-config-volume\") pod \"payload-pre-processing-7964457696-tmjqm\" (UID: \"c6582771-54aa-4c55-b416-3f661db37add\") " pod="openshift-ingress/payload-pre-processing-7964457696-tmjqm" Jul 06 23:59:22.224833 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:22.224834 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9h4m\" (UniqueName: \"kubernetes.io/projected/e9cf329f-b667-4b93-8511-811c1f1efee3-kube-api-access-r9h4m\") pod \"payload-processing-58f86c7798-2bhlm\" (UID: \"e9cf329f-b667-4b93-8511-811c1f1efee3\") " pod="openshift-ingress/payload-processing-58f86c7798-2bhlm" Jul 06 23:59:22.225033 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:22.224870 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52fhz\" (UniqueName: \"kubernetes.io/projected/c6582771-54aa-4c55-b416-3f661db37add-kube-api-access-52fhz\") pod \"payload-pre-processing-7964457696-tmjqm\" (UID: \"c6582771-54aa-4c55-b416-3f661db37add\") " pod="openshift-ingress/payload-pre-processing-7964457696-tmjqm" Jul 06 23:59:22.225033 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:22.224910 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9cf329f-b667-4b93-8511-811c1f1efee3-plugins-config-volume\") pod \"payload-processing-58f86c7798-2bhlm\" (UID: \"e9cf329f-b667-4b93-8511-811c1f1efee3\") " pod="openshift-ingress/payload-processing-58f86c7798-2bhlm" Jul 06 23:59:22.325889 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:22.325856 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6582771-54aa-4c55-b416-3f661db37add-plugins-config-volume\") pod \"payload-pre-processing-7964457696-tmjqm\" (UID: \"c6582771-54aa-4c55-b416-3f661db37add\") " pod="openshift-ingress/payload-pre-processing-7964457696-tmjqm" Jul 06 23:59:22.326070 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:22.325895 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9h4m\" (UniqueName: \"kubernetes.io/projected/e9cf329f-b667-4b93-8511-811c1f1efee3-kube-api-access-r9h4m\") pod \"payload-processing-58f86c7798-2bhlm\" (UID: \"e9cf329f-b667-4b93-8511-811c1f1efee3\") " pod="openshift-ingress/payload-processing-58f86c7798-2bhlm" Jul 06 23:59:22.326070 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:22.325947 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52fhz\" (UniqueName: \"kubernetes.io/projected/c6582771-54aa-4c55-b416-3f661db37add-kube-api-access-52fhz\") pod \"payload-pre-processing-7964457696-tmjqm\" (UID: \"c6582771-54aa-4c55-b416-3f661db37add\") " pod="openshift-ingress/payload-pre-processing-7964457696-tmjqm" Jul 06 23:59:22.326070 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:22.325993 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9cf329f-b667-4b93-8511-811c1f1efee3-plugins-config-volume\") pod \"payload-processing-58f86c7798-2bhlm\" (UID: \"e9cf329f-b667-4b93-8511-811c1f1efee3\") " pod="openshift-ingress/payload-processing-58f86c7798-2bhlm" Jul 06 23:59:22.326820 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:22.326790 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9cf329f-b667-4b93-8511-811c1f1efee3-plugins-config-volume\") pod \"payload-processing-58f86c7798-2bhlm\" (UID: \"e9cf329f-b667-4b93-8511-811c1f1efee3\") " pod="openshift-ingress/payload-processing-58f86c7798-2bhlm" Jul 06 23:59:22.326938 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:22.326901 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6582771-54aa-4c55-b416-3f661db37add-plugins-config-volume\") pod \"payload-pre-processing-7964457696-tmjqm\" (UID: \"c6582771-54aa-4c55-b416-3f661db37add\") " pod="openshift-ingress/payload-pre-processing-7964457696-tmjqm" Jul 06 23:59:22.334403 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:22.334371 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52fhz\" (UniqueName: \"kubernetes.io/projected/c6582771-54aa-4c55-b416-3f661db37add-kube-api-access-52fhz\") pod \"payload-pre-processing-7964457696-tmjqm\" (UID: \"c6582771-54aa-4c55-b416-3f661db37add\") " pod="openshift-ingress/payload-pre-processing-7964457696-tmjqm" Jul 06 23:59:22.334538 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:22.334436 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9h4m\" (UniqueName: \"kubernetes.io/projected/e9cf329f-b667-4b93-8511-811c1f1efee3-kube-api-access-r9h4m\") pod \"payload-processing-58f86c7798-2bhlm\" (UID: \"e9cf329f-b667-4b93-8511-811c1f1efee3\") " pod="openshift-ingress/payload-processing-58f86c7798-2bhlm" Jul 06 23:59:22.466342 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:22.466206 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-tmjqm" Jul 06 23:59:22.512490 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:22.512452 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-2bhlm" Jul 06 23:59:22.579227 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:22.578355 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-7bdf95f944-bdxls" event={"ID":"818a2117-421b-478b-9633-a95a0a927158","Type":"ContainerStarted","Data":"242a4f6d3cfa99660829033a03c25fe008b97710f1d6d5094ea9fc23996d49d8"} Jul 06 23:59:22.641382 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:59:22.641343 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6582771_54aa_4c55_b416_3f661db37add.slice/crio-cee9c7526881c023acc1e92c1e4f63691bfee19c6d6ebf18a72dc100d59cd377 WatchSource:0}: Error finding container cee9c7526881c023acc1e92c1e4f63691bfee19c6d6ebf18a72dc100d59cd377: Status 404 returned error can't find the container with id cee9c7526881c023acc1e92c1e4f63691bfee19c6d6ebf18a72dc100d59cd377 Jul 06 23:59:22.642421 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:22.641557 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-tmjqm"] Jul 06 23:59:22.704234 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:22.704209 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-2bhlm"] Jul 06 23:59:22.705865 ip-10-0-136-25 kubenswrapper[2553]: W0706 23:59:22.705842 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9cf329f_b667_4b93_8511_811c1f1efee3.slice/crio-e3cb4125642ec1e58ac0d11d43ac860d0ce1d0c8b43381e0659d7d86d17c4e34 WatchSource:0}: Error finding container e3cb4125642ec1e58ac0d11d43ac860d0ce1d0c8b43381e0659d7d86d17c4e34: Status 404 returned error can't find the container with id e3cb4125642ec1e58ac0d11d43ac860d0ce1d0c8b43381e0659d7d86d17c4e34 Jul 06 23:59:23.590907 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:23.590814 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-2bhlm" event={"ID":"e9cf329f-b667-4b93-8511-811c1f1efee3","Type":"ContainerStarted","Data":"e3cb4125642ec1e58ac0d11d43ac860d0ce1d0c8b43381e0659d7d86d17c4e34"} Jul 06 23:59:23.592696 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:23.592655 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-tmjqm" event={"ID":"c6582771-54aa-4c55-b416-3f661db37add","Type":"ContainerStarted","Data":"cee9c7526881c023acc1e92c1e4f63691bfee19c6d6ebf18a72dc100d59cd377"} Jul 06 23:59:25.605830 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:25.605790 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-2bhlm" event={"ID":"e9cf329f-b667-4b93-8511-811c1f1efee3","Type":"ContainerStarted","Data":"82ae56c1951f19989ca93f4560cdac31c2a14eb2c1e5b66def3a2b64fb75a701"} Jul 06 23:59:25.606360 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:25.605983 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-2bhlm" Jul 06 23:59:25.607501 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:25.607480 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-7bdf95f944-bdxls" event={"ID":"818a2117-421b-478b-9633-a95a0a927158","Type":"ContainerStarted","Data":"3435edc64110ab044a9270ec6afd3e23c43cb7f987663d616ecc6a5b91ec0e6b"} Jul 06 23:59:25.607607 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:25.607562 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-7bdf95f944-bdxls" Jul 06 23:59:25.608994 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:25.608970 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-tmjqm" event={"ID":"c6582771-54aa-4c55-b416-3f661db37add","Type":"ContainerStarted","Data":"5290225c773ef9092ac76921d326b155979270093bda211520e7af656cadda10"} Jul 06 23:59:25.609111 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:25.609095 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-tmjqm" Jul 06 23:59:25.622268 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:25.622200 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-2bhlm" podStartSLOduration=1.7536612150000002 podStartE2EDuration="3.622178096s" totalImagesPullingTime="1.868516881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:59:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:59:25.621473454 +0000 UTC m=+647.831202976" watchObservedRunningTime="2026-07-06 23:59:25.622178096 +0000 UTC m=+647.831907598" Jul 06 23:59:25.638457 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:25.638390 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-7bdf95f944-bdxls" podStartSLOduration=2.25498331 podStartE2EDuration="4.638364998s" totalImagesPullingTime="2.383381688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:59:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:59:25.636511946 +0000 UTC m=+647.846241505" watchObservedRunningTime="2026-07-06 23:59:25.638364998 +0000 UTC m=+647.848094520" Jul 06 23:59:25.652919 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:25.652861 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-tmjqm" podStartSLOduration=1.717825392 podStartE2EDuration="3.652843013s" totalImagesPullingTime="1.935017621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:59:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:59:25.651821784 +0000 UTC m=+647.861551364" watchObservedRunningTime="2026-07-06 23:59:25.652843013 +0000 UTC m=+647.862572533" Jul 06 23:59:29.646540 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:29.646508 2553 generic.go:356] "Generic (PLEG): container finished" podID="867fe4ad-91d7-4301-b82a-f5eaf31f863d" containerID="f00637479020d544420867a4ea2954febc30213fd1e551367e3635ebcbc958f6" exitCode=0 Jul 06 23:59:29.646957 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:29.646590 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" event={"ID":"867fe4ad-91d7-4301-b82a-f5eaf31f863d","Type":"ContainerDied","Data":"f00637479020d544420867a4ea2954febc30213fd1e551367e3635ebcbc958f6"} Jul 06 23:59:29.647013 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:29.646999 2553 scope.go:117] "RemoveContainer" containerID="f00637479020d544420867a4ea2954febc30213fd1e551367e3635ebcbc958f6" Jul 06 23:59:29.648014 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:29.647985 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 23:59:29.648124 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:29.648076 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 23:59:29.648192 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:29.648125 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 23:59:30.654150 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:30.654115 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" event={"ID":"867fe4ad-91d7-4301-b82a-f5eaf31f863d","Type":"ContainerStarted","Data":"1a913b84ac934de1e28fa2f509746640ba7c008a39fc8174f2c573052c4c9857"} Jul 06 23:59:30.699034 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:30.698993 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:59:30.699248 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:30.699226 2553 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-7x84r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 06 23:59:30.699335 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:30.699273 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" podUID="867fe4ad-91d7-4301-b82a-f5eaf31f863d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 06 23:59:31.641626 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:31.641590 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-7bdf95f944-bdxls" Jul 06 23:59:31.699324 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:31.699276 2553 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-7x84r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 06 23:59:31.699706 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:31.699369 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" podUID="867fe4ad-91d7-4301-b82a-f5eaf31f863d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 06 23:59:32.699380 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:32.699335 2553 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-7x84r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 06 23:59:32.699803 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:32.699398 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" podUID="867fe4ad-91d7-4301-b82a-f5eaf31f863d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 06 23:59:33.699242 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:33.699198 2553 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-7x84r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 06 23:59:33.699488 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:33.699268 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" podUID="867fe4ad-91d7-4301-b82a-f5eaf31f863d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 06 23:59:34.702744 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:34.702711 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:59:34.703221 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:34.703092 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:59:34.703725 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:34.703708 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-7x84r" Jul 06 23:59:36.628233 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:36.628193 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-7964457696-tmjqm" podUID="c6582771-54aa-4c55-b416-3f661db37add" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.58:9004: connect: connection refused" Jul 06 23:59:36.628675 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:36.628539 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-2bhlm" Jul 06 23:59:46.627467 ip-10-0-136-25 kubenswrapper[2553]: I0706 23:59:46.627442 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-tmjqm" Jul 07 00:00:00.165219 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:00.165181 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29723040-s84z8"] Jul 07 00:00:00.177974 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:00.177935 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723040-s84z8" Jul 07 00:00:00.187117 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:00.187086 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723040-s84z8"] Jul 07 00:00:00.285591 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:00.285557 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-pruner-29723040-zbd4s"] Jul 07 00:00:00.289610 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:00.289586 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29723040-zbd4s" Jul 07 00:00:00.295511 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:00.295483 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"pruner-dockercfg-b4626\"" Jul 07 00:00:00.295743 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:00.295673 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"serviceca\"" Jul 07 00:00:00.298706 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:00.298681 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg29q\" (UniqueName: \"kubernetes.io/projected/34d89dc6-76e4-42c1-846a-6708ea0d87ea-kube-api-access-mg29q\") pod \"maas-api-key-cleanup-29723040-s84z8\" (UID: \"34d89dc6-76e4-42c1-846a-6708ea0d87ea\") " pod="opendatahub/maas-api-key-cleanup-29723040-s84z8" Jul 07 00:00:00.319279 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:00.319244 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29723040-zbd4s"] Jul 07 00:00:00.399735 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:00.399692 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgrbj\" (UniqueName: \"kubernetes.io/projected/84a468e6-58a8-4f68-8a2b-929c319ec7e0-kube-api-access-xgrbj\") pod \"image-pruner-29723040-zbd4s\" (UID: \"84a468e6-58a8-4f68-8a2b-929c319ec7e0\") " pod="openshift-image-registry/image-pruner-29723040-zbd4s" Jul 07 00:00:00.399735 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:00.399741 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/84a468e6-58a8-4f68-8a2b-929c319ec7e0-serviceca\") pod \"image-pruner-29723040-zbd4s\" (UID: \"84a468e6-58a8-4f68-8a2b-929c319ec7e0\") " pod="openshift-image-registry/image-pruner-29723040-zbd4s" Jul 07 00:00:00.399959 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:00.399900 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg29q\" (UniqueName: \"kubernetes.io/projected/34d89dc6-76e4-42c1-846a-6708ea0d87ea-kube-api-access-mg29q\") pod \"maas-api-key-cleanup-29723040-s84z8\" (UID: \"34d89dc6-76e4-42c1-846a-6708ea0d87ea\") " pod="opendatahub/maas-api-key-cleanup-29723040-s84z8" Jul 07 00:00:00.409834 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:00.409797 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg29q\" (UniqueName: \"kubernetes.io/projected/34d89dc6-76e4-42c1-846a-6708ea0d87ea-kube-api-access-mg29q\") pod \"maas-api-key-cleanup-29723040-s84z8\" (UID: \"34d89dc6-76e4-42c1-846a-6708ea0d87ea\") " pod="opendatahub/maas-api-key-cleanup-29723040-s84z8" Jul 07 00:00:00.501161 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:00.501070 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgrbj\" (UniqueName: \"kubernetes.io/projected/84a468e6-58a8-4f68-8a2b-929c319ec7e0-kube-api-access-xgrbj\") pod \"image-pruner-29723040-zbd4s\" (UID: \"84a468e6-58a8-4f68-8a2b-929c319ec7e0\") " pod="openshift-image-registry/image-pruner-29723040-zbd4s" Jul 07 00:00:00.501161 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:00.501114 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/84a468e6-58a8-4f68-8a2b-929c319ec7e0-serviceca\") pod \"image-pruner-29723040-zbd4s\" (UID: \"84a468e6-58a8-4f68-8a2b-929c319ec7e0\") " pod="openshift-image-registry/image-pruner-29723040-zbd4s" Jul 07 00:00:00.501863 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:00.501844 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/84a468e6-58a8-4f68-8a2b-929c319ec7e0-serviceca\") pod \"image-pruner-29723040-zbd4s\" (UID: \"84a468e6-58a8-4f68-8a2b-929c319ec7e0\") " pod="openshift-image-registry/image-pruner-29723040-zbd4s" Jul 07 00:00:00.507893 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:00.507865 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723040-s84z8" Jul 07 00:00:00.512597 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:00.512577 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgrbj\" (UniqueName: \"kubernetes.io/projected/84a468e6-58a8-4f68-8a2b-929c319ec7e0-kube-api-access-xgrbj\") pod \"image-pruner-29723040-zbd4s\" (UID: \"84a468e6-58a8-4f68-8a2b-929c319ec7e0\") " pod="openshift-image-registry/image-pruner-29723040-zbd4s" Jul 07 00:00:00.599777 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:00.599736 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29723040-zbd4s" Jul 07 00:00:00.677994 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:00.676742 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723040-s84z8"] Jul 07 00:00:00.760331 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:00.760286 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29723040-zbd4s"] Jul 07 00:00:00.762086 ip-10-0-136-25 kubenswrapper[2553]: W0707 00:00:00.762061 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84a468e6_58a8_4f68_8a2b_929c319ec7e0.slice/crio-a65f45b7629727b3aa3e4266bd28c95fb58b96c94d575abf4074706828759428 WatchSource:0}: Error finding container a65f45b7629727b3aa3e4266bd28c95fb58b96c94d575abf4074706828759428: Status 404 returned error can't find the container with id a65f45b7629727b3aa3e4266bd28c95fb58b96c94d575abf4074706828759428 Jul 07 00:00:00.824193 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:00.824164 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29723040-zbd4s" event={"ID":"84a468e6-58a8-4f68-8a2b-929c319ec7e0","Type":"ContainerStarted","Data":"a65f45b7629727b3aa3e4266bd28c95fb58b96c94d575abf4074706828759428"} Jul 07 00:00:00.825505 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:00.825473 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723040-s84z8" event={"ID":"34d89dc6-76e4-42c1-846a-6708ea0d87ea","Type":"ContainerStarted","Data":"c127231f2282ddeeafa2dd30373a78f963df3ddcfab6d3b759469c9afd0ca246"} Jul 07 00:00:01.496639 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:01.496560 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft"] Jul 07 00:00:01.500456 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:01.500433 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft" Jul 07 00:00:01.503981 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:01.503951 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 00:00:01.504128 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:01.503967 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 00:00:01.505341 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:01.505320 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 07 00:00:01.505428 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:01.505359 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4pfpg\"" Jul 07 00:00:01.514674 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:01.514645 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft"] Jul 07 00:00:01.612787 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:01.612753 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41218329-f2c7-4623-b636-c99ec3352819-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkbft\" (UID: \"41218329-f2c7-4623-b636-c99ec3352819\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft" Jul 07 00:00:01.613019 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:01.612881 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41218329-f2c7-4623-b636-c99ec3352819-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkbft\" (UID: \"41218329-f2c7-4623-b636-c99ec3352819\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft" Jul 07 00:00:01.613019 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:01.612909 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41218329-f2c7-4623-b636-c99ec3352819-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkbft\" (UID: \"41218329-f2c7-4623-b636-c99ec3352819\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft" Jul 07 00:00:01.613019 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:01.612966 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41218329-f2c7-4623-b636-c99ec3352819-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkbft\" (UID: \"41218329-f2c7-4623-b636-c99ec3352819\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft" Jul 07 00:00:01.613019 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:01.612990 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6tqd\" (UniqueName: \"kubernetes.io/projected/41218329-f2c7-4623-b636-c99ec3352819-kube-api-access-q6tqd\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkbft\" (UID: \"41218329-f2c7-4623-b636-c99ec3352819\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft" Jul 07 00:00:01.613019 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:01.613015 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41218329-f2c7-4623-b636-c99ec3352819-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkbft\" (UID: \"41218329-f2c7-4623-b636-c99ec3352819\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft" Jul 07 00:00:01.714472 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:01.714433 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41218329-f2c7-4623-b636-c99ec3352819-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkbft\" (UID: \"41218329-f2c7-4623-b636-c99ec3352819\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft" Jul 07 00:00:01.714472 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:01.714473 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41218329-f2c7-4623-b636-c99ec3352819-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkbft\" (UID: \"41218329-f2c7-4623-b636-c99ec3352819\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft" Jul 07 00:00:01.714727 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:01.714505 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41218329-f2c7-4623-b636-c99ec3352819-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkbft\" (UID: \"41218329-f2c7-4623-b636-c99ec3352819\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft" Jul 07 00:00:01.714727 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:01.714617 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6tqd\" (UniqueName: \"kubernetes.io/projected/41218329-f2c7-4623-b636-c99ec3352819-kube-api-access-q6tqd\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkbft\" (UID: \"41218329-f2c7-4623-b636-c99ec3352819\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft" Jul 07 00:00:01.714727 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:01.714660 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41218329-f2c7-4623-b636-c99ec3352819-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkbft\" (UID: \"41218329-f2c7-4623-b636-c99ec3352819\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft" Jul 07 00:00:01.714727 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:01.714711 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41218329-f2c7-4623-b636-c99ec3352819-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkbft\" (UID: \"41218329-f2c7-4623-b636-c99ec3352819\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft" Jul 07 00:00:01.714928 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:01.714871 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41218329-f2c7-4623-b636-c99ec3352819-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkbft\" (UID: \"41218329-f2c7-4623-b636-c99ec3352819\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft" Jul 07 00:00:01.714928 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:01.714907 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41218329-f2c7-4623-b636-c99ec3352819-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkbft\" (UID: \"41218329-f2c7-4623-b636-c99ec3352819\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft" Jul 07 00:00:01.715166 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:01.715138 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41218329-f2c7-4623-b636-c99ec3352819-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkbft\" (UID: \"41218329-f2c7-4623-b636-c99ec3352819\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft" Jul 07 00:00:01.716947 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:01.716921 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41218329-f2c7-4623-b636-c99ec3352819-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkbft\" (UID: \"41218329-f2c7-4623-b636-c99ec3352819\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft" Jul 07 00:00:01.717272 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:01.717253 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41218329-f2c7-4623-b636-c99ec3352819-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkbft\" (UID: \"41218329-f2c7-4623-b636-c99ec3352819\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft" Jul 07 00:00:01.724004 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:01.723975 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6tqd\" (UniqueName: \"kubernetes.io/projected/41218329-f2c7-4623-b636-c99ec3352819-kube-api-access-q6tqd\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkbft\" (UID: \"41218329-f2c7-4623-b636-c99ec3352819\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft" Jul 07 00:00:01.813892 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:01.813794 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft" Jul 07 00:00:01.834706 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:01.834674 2553 generic.go:356] "Generic (PLEG): container finished" podID="84a468e6-58a8-4f68-8a2b-929c319ec7e0" containerID="f809560af316c4b71f5d2fca46ce506facea0d222a9f1d43edc645d652bc7c0e" exitCode=0 Jul 07 00:00:01.834872 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:01.834757 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29723040-zbd4s" event={"ID":"84a468e6-58a8-4f68-8a2b-929c319ec7e0","Type":"ContainerDied","Data":"f809560af316c4b71f5d2fca46ce506facea0d222a9f1d43edc645d652bc7c0e"} Jul 07 00:00:01.965883 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:01.965854 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft"] Jul 07 00:00:02.841718 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:02.841676 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft" event={"ID":"41218329-f2c7-4623-b636-c99ec3352819","Type":"ContainerStarted","Data":"40717cec3bcdc61ab16f4d52849a598acbd18ba95ca3a56a66983cb59a4e09c4"} Jul 07 00:00:02.997867 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:02.997835 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29723040-zbd4s" Jul 07 00:00:03.129896 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:03.129795 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/84a468e6-58a8-4f68-8a2b-929c319ec7e0-serviceca\") pod \"84a468e6-58a8-4f68-8a2b-929c319ec7e0\" (UID: \"84a468e6-58a8-4f68-8a2b-929c319ec7e0\") " Jul 07 00:00:03.129896 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:03.129844 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgrbj\" (UniqueName: \"kubernetes.io/projected/84a468e6-58a8-4f68-8a2b-929c319ec7e0-kube-api-access-xgrbj\") pod \"84a468e6-58a8-4f68-8a2b-929c319ec7e0\" (UID: \"84a468e6-58a8-4f68-8a2b-929c319ec7e0\") " Jul 07 00:00:03.130200 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:03.130177 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84a468e6-58a8-4f68-8a2b-929c319ec7e0-serviceca" (OuterVolumeSpecName: "serviceca") pod "84a468e6-58a8-4f68-8a2b-929c319ec7e0" (UID: "84a468e6-58a8-4f68-8a2b-929c319ec7e0"). InnerVolumeSpecName "serviceca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:00:03.132201 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:03.132172 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84a468e6-58a8-4f68-8a2b-929c319ec7e0-kube-api-access-xgrbj" (OuterVolumeSpecName: "kube-api-access-xgrbj") pod "84a468e6-58a8-4f68-8a2b-929c319ec7e0" (UID: "84a468e6-58a8-4f68-8a2b-929c319ec7e0"). InnerVolumeSpecName "kube-api-access-xgrbj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:00:03.231451 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:03.231409 2553 reconciler_common.go:299] "Volume detached for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/84a468e6-58a8-4f68-8a2b-929c319ec7e0-serviceca\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:00:03.231451 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:03.231452 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgrbj\" (UniqueName: \"kubernetes.io/projected/84a468e6-58a8-4f68-8a2b-929c319ec7e0-kube-api-access-xgrbj\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:00:03.848609 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:03.848571 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29723040-zbd4s" event={"ID":"84a468e6-58a8-4f68-8a2b-929c319ec7e0","Type":"ContainerDied","Data":"a65f45b7629727b3aa3e4266bd28c95fb58b96c94d575abf4074706828759428"} Jul 07 00:00:03.848609 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:03.848601 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29723040-zbd4s" Jul 07 00:00:03.849032 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:03.848628 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a65f45b7629727b3aa3e4266bd28c95fb58b96c94d575abf4074706828759428" Jul 07 00:00:05.863738 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:05.863645 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723040-s84z8" event={"ID":"34d89dc6-76e4-42c1-846a-6708ea0d87ea","Type":"ContainerStarted","Data":"dd2726a0de0dcccc5315665efa2e84d78061c7dcbc91a390115b132feaa5ff43"} Jul 07 00:00:05.880489 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:05.880440 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29723040-s84z8" podStartSLOduration=0.992838144 podStartE2EDuration="5.880424721s" totalImagesPullingTime="4.887586577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:00:05.878549566 +0000 UTC m=+688.088279110" watchObservedRunningTime="2026-07-07 00:00:05.880424721 +0000 UTC m=+688.090154242" Jul 07 00:00:06.870771 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:06.870676 2553 generic.go:356] "Generic (PLEG): container finished" podID="34d89dc6-76e4-42c1-846a-6708ea0d87ea" containerID="dd2726a0de0dcccc5315665efa2e84d78061c7dcbc91a390115b132feaa5ff43" exitCode=0 Jul 07 00:00:06.871131 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:06.870766 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723040-s84z8" event={"ID":"34d89dc6-76e4-42c1-846a-6708ea0d87ea","Type":"ContainerDied","Data":"dd2726a0de0dcccc5315665efa2e84d78061c7dcbc91a390115b132feaa5ff43"} Jul 07 00:00:08.019817 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:08.019787 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723040-s84z8" Jul 07 00:00:08.073048 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:08.073017 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mg29q\" (UniqueName: \"kubernetes.io/projected/34d89dc6-76e4-42c1-846a-6708ea0d87ea-kube-api-access-mg29q\") pod \"34d89dc6-76e4-42c1-846a-6708ea0d87ea\" (UID: \"34d89dc6-76e4-42c1-846a-6708ea0d87ea\") " Jul 07 00:00:08.075079 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:08.075048 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34d89dc6-76e4-42c1-846a-6708ea0d87ea-kube-api-access-mg29q" (OuterVolumeSpecName: "kube-api-access-mg29q") pod "34d89dc6-76e4-42c1-846a-6708ea0d87ea" (UID: "34d89dc6-76e4-42c1-846a-6708ea0d87ea"). InnerVolumeSpecName "kube-api-access-mg29q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:00:08.173836 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:08.173803 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mg29q\" (UniqueName: \"kubernetes.io/projected/34d89dc6-76e4-42c1-846a-6708ea0d87ea-kube-api-access-mg29q\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:00:08.882422 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:08.882394 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723040-s84z8" Jul 07 00:00:08.882603 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:08.882440 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723040-s84z8" event={"ID":"34d89dc6-76e4-42c1-846a-6708ea0d87ea","Type":"ContainerDied","Data":"c127231f2282ddeeafa2dd30373a78f963df3ddcfab6d3b759469c9afd0ca246"} Jul 07 00:00:08.882603 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:08.882487 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c127231f2282ddeeafa2dd30373a78f963df3ddcfab6d3b759469c9afd0ca246" Jul 07 00:00:11.902187 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:11.902151 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft" event={"ID":"41218329-f2c7-4623-b636-c99ec3352819","Type":"ContainerStarted","Data":"d945c4a54c7197efb54c2d63916bf0b21cd782e08e477cb019cb681bef667447"} Jul 07 00:00:11.922969 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:11.922914 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft" podStartSLOduration=1.2999326789999999 podStartE2EDuration="10.922896788s" totalImagesPullingTime="9.622964109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:00:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:00:11.919208819 +0000 UTC m=+694.128938370" watchObservedRunningTime="2026-07-07 00:00:11.922896788 +0000 UTC m=+694.132626308" Jul 07 00:00:21.814341 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:21.814270 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft" Jul 07 00:00:21.814813 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:21.814374 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft" Jul 07 00:00:21.828433 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:21.828389 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft" Jul 07 00:00:21.973725 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:21.973692 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkbft" Jul 07 00:00:24.880887 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:24.880847 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff"] Jul 07 00:00:24.881524 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:24.881504 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34d89dc6-76e4-42c1-846a-6708ea0d87ea" containerName="cleanup" Jul 07 00:00:24.881622 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:24.881527 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="34d89dc6-76e4-42c1-846a-6708ea0d87ea" containerName="cleanup" Jul 07 00:00:24.881622 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:24.881562 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84a468e6-58a8-4f68-8a2b-929c319ec7e0" containerName="image-pruner" Jul 07 00:00:24.881622 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:24.881571 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="84a468e6-58a8-4f68-8a2b-929c319ec7e0" containerName="image-pruner" Jul 07 00:00:24.881769 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:24.881675 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="84a468e6-58a8-4f68-8a2b-929c319ec7e0" containerName="image-pruner" Jul 07 00:00:24.881769 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:24.881689 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="34d89dc6-76e4-42c1-846a-6708ea0d87ea" containerName="cleanup" Jul 07 00:00:24.885141 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:24.885117 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff" Jul 07 00:00:24.888004 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:24.887982 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 07 00:00:24.896816 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:24.896787 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff"] Jul 07 00:00:24.936227 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:24.936193 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5f3159e-717f-4263-8ce9-b317ecb5d8c4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4dhff\" (UID: \"b5f3159e-717f-4263-8ce9-b317ecb5d8c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff" Jul 07 00:00:24.936429 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:24.936252 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5f3159e-717f-4263-8ce9-b317ecb5d8c4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4dhff\" (UID: \"b5f3159e-717f-4263-8ce9-b317ecb5d8c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff" Jul 07 00:00:24.936429 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:24.936333 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5f3159e-717f-4263-8ce9-b317ecb5d8c4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4dhff\" (UID: \"b5f3159e-717f-4263-8ce9-b317ecb5d8c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff" Jul 07 00:00:24.936429 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:24.936366 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5f3159e-717f-4263-8ce9-b317ecb5d8c4-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4dhff\" (UID: \"b5f3159e-717f-4263-8ce9-b317ecb5d8c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff" Jul 07 00:00:24.936429 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:24.936393 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9cdv\" (UniqueName: \"kubernetes.io/projected/b5f3159e-717f-4263-8ce9-b317ecb5d8c4-kube-api-access-m9cdv\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4dhff\" (UID: \"b5f3159e-717f-4263-8ce9-b317ecb5d8c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff" Jul 07 00:00:24.936572 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:24.936431 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5f3159e-717f-4263-8ce9-b317ecb5d8c4-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4dhff\" (UID: \"b5f3159e-717f-4263-8ce9-b317ecb5d8c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff" Jul 07 00:00:25.037108 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:25.037069 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5f3159e-717f-4263-8ce9-b317ecb5d8c4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4dhff\" (UID: \"b5f3159e-717f-4263-8ce9-b317ecb5d8c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff" Jul 07 00:00:25.037338 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:25.037220 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5f3159e-717f-4263-8ce9-b317ecb5d8c4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4dhff\" (UID: \"b5f3159e-717f-4263-8ce9-b317ecb5d8c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff" Jul 07 00:00:25.037338 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:25.037245 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5f3159e-717f-4263-8ce9-b317ecb5d8c4-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4dhff\" (UID: \"b5f3159e-717f-4263-8ce9-b317ecb5d8c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff" Jul 07 00:00:25.037338 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:25.037277 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9cdv\" (UniqueName: \"kubernetes.io/projected/b5f3159e-717f-4263-8ce9-b317ecb5d8c4-kube-api-access-m9cdv\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4dhff\" (UID: \"b5f3159e-717f-4263-8ce9-b317ecb5d8c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff" Jul 07 00:00:25.037516 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:25.037358 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5f3159e-717f-4263-8ce9-b317ecb5d8c4-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4dhff\" (UID: \"b5f3159e-717f-4263-8ce9-b317ecb5d8c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff" Jul 07 00:00:25.037516 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:25.037412 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5f3159e-717f-4263-8ce9-b317ecb5d8c4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4dhff\" (UID: \"b5f3159e-717f-4263-8ce9-b317ecb5d8c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff" Jul 07 00:00:25.037734 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:25.037651 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5f3159e-717f-4263-8ce9-b317ecb5d8c4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4dhff\" (UID: \"b5f3159e-717f-4263-8ce9-b317ecb5d8c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff" Jul 07 00:00:25.037734 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:25.037710 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5f3159e-717f-4263-8ce9-b317ecb5d8c4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4dhff\" (UID: \"b5f3159e-717f-4263-8ce9-b317ecb5d8c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff" Jul 07 00:00:25.037872 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:25.037744 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5f3159e-717f-4263-8ce9-b317ecb5d8c4-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4dhff\" (UID: \"b5f3159e-717f-4263-8ce9-b317ecb5d8c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff" Jul 07 00:00:25.039798 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:25.039768 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5f3159e-717f-4263-8ce9-b317ecb5d8c4-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4dhff\" (UID: \"b5f3159e-717f-4263-8ce9-b317ecb5d8c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff" Jul 07 00:00:25.040047 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:25.040030 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5f3159e-717f-4263-8ce9-b317ecb5d8c4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4dhff\" (UID: \"b5f3159e-717f-4263-8ce9-b317ecb5d8c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff" Jul 07 00:00:25.046540 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:25.046509 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9cdv\" (UniqueName: \"kubernetes.io/projected/b5f3159e-717f-4263-8ce9-b317ecb5d8c4-kube-api-access-m9cdv\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4dhff\" (UID: \"b5f3159e-717f-4263-8ce9-b317ecb5d8c4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff" Jul 07 00:00:25.196956 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:25.196916 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff" Jul 07 00:00:25.354116 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:25.354087 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff"] Jul 07 00:00:25.355203 ip-10-0-136-25 kubenswrapper[2553]: W0707 00:00:25.355178 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5f3159e_717f_4263_8ce9_b317ecb5d8c4.slice/crio-39e0daedf3616d984cb96ebb552439685c00c6389b2be5cba9b69f2335c2b60b WatchSource:0}: Error finding container 39e0daedf3616d984cb96ebb552439685c00c6389b2be5cba9b69f2335c2b60b: Status 404 returned error can't find the container with id 39e0daedf3616d984cb96ebb552439685c00c6389b2be5cba9b69f2335c2b60b Jul 07 00:00:25.986146 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:25.986106 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff" event={"ID":"b5f3159e-717f-4263-8ce9-b317ecb5d8c4","Type":"ContainerStarted","Data":"16e8f7c6dd998bfed929919634f6688b3f778d6976fd9eb8dc360fd7dc7c1858"} Jul 07 00:00:25.986597 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:25.986172 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff" event={"ID":"b5f3159e-717f-4263-8ce9-b317ecb5d8c4","Type":"ContainerStarted","Data":"39e0daedf3616d984cb96ebb552439685c00c6389b2be5cba9b69f2335c2b60b"} Jul 07 00:00:26.006014 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:26.005958 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff" podStartSLOduration=1.808836875 podStartE2EDuration="2.005940667s" totalImagesPullingTime="197.103792ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:00:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:00:26.003371732 +0000 UTC m=+708.213101253" watchObservedRunningTime="2026-07-07 00:00:26.005940667 +0000 UTC m=+708.215670188" Jul 07 00:00:35.197131 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:35.197031 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff" Jul 07 00:00:35.197680 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:35.197147 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff" Jul 07 00:00:35.209801 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:35.209772 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff" Jul 07 00:00:36.058153 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:00:36.058123 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4dhff" Jul 07 00:02:50.181138 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:50.181094 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56bf7f957f-llwjm"] Jul 07 00:02:50.181831 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:50.181342 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-56bf7f957f-llwjm" podUID="1c66b153-6263-4f23-998b-e8f4ef77a9dd" containerName="manager" containerID="cri-o://dcdca9451e40b58f87156894b45eff85f0590d9377596150c9ddf31da7ec0e9b" gracePeriod=30 Jul 07 00:02:50.455370 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:50.455342 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56bf7f957f-llwjm" Jul 07 00:02:50.639274 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:50.639238 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fgn2q\" (UniqueName: \"kubernetes.io/projected/1c66b153-6263-4f23-998b-e8f4ef77a9dd-kube-api-access-fgn2q\") pod \"1c66b153-6263-4f23-998b-e8f4ef77a9dd\" (UID: \"1c66b153-6263-4f23-998b-e8f4ef77a9dd\") " Jul 07 00:02:50.639524 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:50.639402 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1c66b153-6263-4f23-998b-e8f4ef77a9dd-webhook-cert\") pod \"1c66b153-6263-4f23-998b-e8f4ef77a9dd\" (UID: \"1c66b153-6263-4f23-998b-e8f4ef77a9dd\") " Jul 07 00:02:50.641661 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:50.641615 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c66b153-6263-4f23-998b-e8f4ef77a9dd-kube-api-access-fgn2q" (OuterVolumeSpecName: "kube-api-access-fgn2q") pod "1c66b153-6263-4f23-998b-e8f4ef77a9dd" (UID: "1c66b153-6263-4f23-998b-e8f4ef77a9dd"). InnerVolumeSpecName "kube-api-access-fgn2q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:02:50.641810 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:50.641664 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c66b153-6263-4f23-998b-e8f4ef77a9dd-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "1c66b153-6263-4f23-998b-e8f4ef77a9dd" (UID: "1c66b153-6263-4f23-998b-e8f4ef77a9dd"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 00:02:50.741219 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:50.741116 2553 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1c66b153-6263-4f23-998b-e8f4ef77a9dd-webhook-cert\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:02:50.741219 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:50.741160 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgn2q\" (UniqueName: \"kubernetes.io/projected/1c66b153-6263-4f23-998b-e8f4ef77a9dd-kube-api-access-fgn2q\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:02:50.842375 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:50.842341 2553 generic.go:356] "Generic (PLEG): container finished" podID="1c66b153-6263-4f23-998b-e8f4ef77a9dd" containerID="dcdca9451e40b58f87156894b45eff85f0590d9377596150c9ddf31da7ec0e9b" exitCode=0 Jul 07 00:02:50.842564 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:50.842392 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56bf7f957f-llwjm" Jul 07 00:02:50.842564 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:50.842415 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56bf7f957f-llwjm" event={"ID":"1c66b153-6263-4f23-998b-e8f4ef77a9dd","Type":"ContainerDied","Data":"dcdca9451e40b58f87156894b45eff85f0590d9377596150c9ddf31da7ec0e9b"} Jul 07 00:02:50.842564 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:50.842469 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56bf7f957f-llwjm" event={"ID":"1c66b153-6263-4f23-998b-e8f4ef77a9dd","Type":"ContainerDied","Data":"39a148540bbe79d191ca9b1ce20dd315e728880a7b4234826007aa5d2757d04d"} Jul 07 00:02:50.842564 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:50.842483 2553 scope.go:117] "RemoveContainer" containerID="dcdca9451e40b58f87156894b45eff85f0590d9377596150c9ddf31da7ec0e9b" Jul 07 00:02:50.853519 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:50.853495 2553 scope.go:117] "RemoveContainer" containerID="dcdca9451e40b58f87156894b45eff85f0590d9377596150c9ddf31da7ec0e9b" Jul 07 00:02:50.853876 ip-10-0-136-25 kubenswrapper[2553]: E0707 00:02:50.853856 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcdca9451e40b58f87156894b45eff85f0590d9377596150c9ddf31da7ec0e9b\": container with ID starting with dcdca9451e40b58f87156894b45eff85f0590d9377596150c9ddf31da7ec0e9b not found: ID does not exist" containerID="dcdca9451e40b58f87156894b45eff85f0590d9377596150c9ddf31da7ec0e9b" Jul 07 00:02:50.853924 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:50.853886 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcdca9451e40b58f87156894b45eff85f0590d9377596150c9ddf31da7ec0e9b"} err="failed to get container status \"dcdca9451e40b58f87156894b45eff85f0590d9377596150c9ddf31da7ec0e9b\": rpc error: code = NotFound desc = could not find container \"dcdca9451e40b58f87156894b45eff85f0590d9377596150c9ddf31da7ec0e9b\": container with ID starting with dcdca9451e40b58f87156894b45eff85f0590d9377596150c9ddf31da7ec0e9b not found: ID does not exist" Jul 07 00:02:50.867226 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:50.867170 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56bf7f957f-llwjm"] Jul 07 00:02:50.870697 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:50.870662 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-56bf7f957f-llwjm"] Jul 07 00:02:51.630966 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:51.630926 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-56bf7f957f-sjkb7"] Jul 07 00:02:51.631504 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:51.631468 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c66b153-6263-4f23-998b-e8f4ef77a9dd" containerName="manager" Jul 07 00:02:51.631504 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:51.631491 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c66b153-6263-4f23-998b-e8f4ef77a9dd" containerName="manager" Jul 07 00:02:51.631701 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:51.631627 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c66b153-6263-4f23-998b-e8f4ef77a9dd" containerName="manager" Jul 07 00:02:51.636191 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:51.636171 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56bf7f957f-sjkb7" Jul 07 00:02:51.638715 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:51.638697 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 00:02:51.638881 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:51.638858 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 00:02:51.640135 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:51.640118 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-rlhkj\"" Jul 07 00:02:51.644055 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:51.643774 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56bf7f957f-sjkb7"] Jul 07 00:02:51.749806 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:51.749752 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xl4zj\" (UniqueName: \"kubernetes.io/projected/52bd584a-8a0e-462f-9ae5-79c5295eab77-kube-api-access-xl4zj\") pod \"maas-controller-56bf7f957f-sjkb7\" (UID: \"52bd584a-8a0e-462f-9ae5-79c5295eab77\") " pod="opendatahub/maas-controller-56bf7f957f-sjkb7" Jul 07 00:02:51.750025 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:51.749837 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52bd584a-8a0e-462f-9ae5-79c5295eab77-webhook-cert\") pod \"maas-controller-56bf7f957f-sjkb7\" (UID: \"52bd584a-8a0e-462f-9ae5-79c5295eab77\") " pod="opendatahub/maas-controller-56bf7f957f-sjkb7" Jul 07 00:02:51.850916 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:51.850887 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl4zj\" (UniqueName: \"kubernetes.io/projected/52bd584a-8a0e-462f-9ae5-79c5295eab77-kube-api-access-xl4zj\") pod \"maas-controller-56bf7f957f-sjkb7\" (UID: \"52bd584a-8a0e-462f-9ae5-79c5295eab77\") " pod="opendatahub/maas-controller-56bf7f957f-sjkb7" Jul 07 00:02:51.851114 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:51.850926 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52bd584a-8a0e-462f-9ae5-79c5295eab77-webhook-cert\") pod \"maas-controller-56bf7f957f-sjkb7\" (UID: \"52bd584a-8a0e-462f-9ae5-79c5295eab77\") " pod="opendatahub/maas-controller-56bf7f957f-sjkb7" Jul 07 00:02:51.853393 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:51.853372 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52bd584a-8a0e-462f-9ae5-79c5295eab77-webhook-cert\") pod \"maas-controller-56bf7f957f-sjkb7\" (UID: \"52bd584a-8a0e-462f-9ae5-79c5295eab77\") " pod="opendatahub/maas-controller-56bf7f957f-sjkb7" Jul 07 00:02:51.859748 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:51.859722 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xl4zj\" (UniqueName: \"kubernetes.io/projected/52bd584a-8a0e-462f-9ae5-79c5295eab77-kube-api-access-xl4zj\") pod \"maas-controller-56bf7f957f-sjkb7\" (UID: \"52bd584a-8a0e-462f-9ae5-79c5295eab77\") " pod="opendatahub/maas-controller-56bf7f957f-sjkb7" Jul 07 00:02:51.948336 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:51.948279 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56bf7f957f-sjkb7" Jul 07 00:02:52.093966 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:52.093939 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56bf7f957f-sjkb7"] Jul 07 00:02:52.095713 ip-10-0-136-25 kubenswrapper[2553]: W0707 00:02:52.095685 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52bd584a_8a0e_462f_9ae5_79c5295eab77.slice/crio-13b2b55c5864fcb4996e307080459edc774484db568e1eadbfd146818436d022 WatchSource:0}: Error finding container 13b2b55c5864fcb4996e307080459edc774484db568e1eadbfd146818436d022: Status 404 returned error can't find the container with id 13b2b55c5864fcb4996e307080459edc774484db568e1eadbfd146818436d022 Jul 07 00:02:52.304692 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:52.304615 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c66b153-6263-4f23-998b-e8f4ef77a9dd" path="/var/lib/kubelet/pods/1c66b153-6263-4f23-998b-e8f4ef77a9dd/volumes" Jul 07 00:02:52.858062 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:52.858023 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56bf7f957f-sjkb7" event={"ID":"52bd584a-8a0e-462f-9ae5-79c5295eab77","Type":"ContainerStarted","Data":"387c9c5b4d39f36b414a66965ab02323a017358fc78c566c8008f7ac4ce22a34"} Jul 07 00:02:52.858574 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:52.858085 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56bf7f957f-sjkb7" event={"ID":"52bd584a-8a0e-462f-9ae5-79c5295eab77","Type":"ContainerStarted","Data":"13b2b55c5864fcb4996e307080459edc774484db568e1eadbfd146818436d022"} Jul 07 00:02:52.858574 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:52.858136 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-56bf7f957f-sjkb7" Jul 07 00:02:52.894494 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:02:52.894436 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-56bf7f957f-sjkb7" podStartSLOduration=1.356811814 podStartE2EDuration="1.894418004s" totalImagesPullingTime="537.60619ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:02:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:02:52.893144952 +0000 UTC m=+855.102874474" watchObservedRunningTime="2026-07-07 00:02:52.894418004 +0000 UTC m=+855.104147526" Jul 07 00:03:03.868077 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:03:03.868036 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-56bf7f957f-sjkb7" Jul 07 00:03:38.237068 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:03:38.236988 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4z8vq_3253bdfe-5248-4d4c-9864-11660eb042ca/ovn-acl-logging/0.log" Jul 07 00:03:38.245320 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:03:38.245266 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4z8vq_3253bdfe-5248-4d4c-9864-11660eb042ca/ovn-acl-logging/0.log" Jul 07 00:07:35.377671 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:07:35.377633 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7nh2l"] Jul 07 00:07:35.378145 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:07:35.377883 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7nh2l" podUID="92e2d5ca-67d5-46e1-bd03-d236964fe689" containerName="manager" containerID="cri-o://27abde383a0d50314aee8a23652b6dd90ba01142e04d6488a551f741c8ded362" gracePeriod=10 Jul 07 00:07:38.979177 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:07:38.979134 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7nh2l" podUID="92e2d5ca-67d5-46e1-bd03-d236964fe689" containerName="manager" probeResult="failure" output="Get \"http://10.132.0.50:8081/readyz\": dial tcp 10.132.0.50:8081: connect: connection refused" Jul 07 00:07:39.529937 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:07:39.529903 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7nh2l" Jul 07 00:07:39.538743 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:07:39.538709 2553 generic.go:356] "Generic (PLEG): container finished" podID="92e2d5ca-67d5-46e1-bd03-d236964fe689" containerID="27abde383a0d50314aee8a23652b6dd90ba01142e04d6488a551f741c8ded362" exitCode=0 Jul 07 00:07:39.538924 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:07:39.538753 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7nh2l" Jul 07 00:07:39.538924 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:07:39.538795 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7nh2l" event={"ID":"92e2d5ca-67d5-46e1-bd03-d236964fe689","Type":"ContainerDied","Data":"27abde383a0d50314aee8a23652b6dd90ba01142e04d6488a551f741c8ded362"} Jul 07 00:07:39.538924 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:07:39.538848 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7nh2l" event={"ID":"92e2d5ca-67d5-46e1-bd03-d236964fe689","Type":"ContainerDied","Data":"2803e16350ee85bae3c3826a0d212844b7311af484156b534a022b48cd5fbe05"} Jul 07 00:07:39.538924 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:07:39.538870 2553 scope.go:117] "RemoveContainer" containerID="27abde383a0d50314aee8a23652b6dd90ba01142e04d6488a551f741c8ded362" Jul 07 00:07:39.550998 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:07:39.550953 2553 scope.go:117] "RemoveContainer" containerID="27abde383a0d50314aee8a23652b6dd90ba01142e04d6488a551f741c8ded362" Jul 07 00:07:39.551364 ip-10-0-136-25 kubenswrapper[2553]: E0707 00:07:39.551341 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27abde383a0d50314aee8a23652b6dd90ba01142e04d6488a551f741c8ded362\": container with ID starting with 27abde383a0d50314aee8a23652b6dd90ba01142e04d6488a551f741c8ded362 not found: ID does not exist" containerID="27abde383a0d50314aee8a23652b6dd90ba01142e04d6488a551f741c8ded362" Jul 07 00:07:39.551464 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:07:39.551373 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27abde383a0d50314aee8a23652b6dd90ba01142e04d6488a551f741c8ded362"} err="failed to get container status \"27abde383a0d50314aee8a23652b6dd90ba01142e04d6488a551f741c8ded362\": rpc error: code = NotFound desc = could not find container \"27abde383a0d50314aee8a23652b6dd90ba01142e04d6488a551f741c8ded362\": container with ID starting with 27abde383a0d50314aee8a23652b6dd90ba01142e04d6488a551f741c8ded362 not found: ID does not exist" Jul 07 00:07:39.571351 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:07:39.571324 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/92e2d5ca-67d5-46e1-bd03-d236964fe689-extensions-socket-volume\") pod \"92e2d5ca-67d5-46e1-bd03-d236964fe689\" (UID: \"92e2d5ca-67d5-46e1-bd03-d236964fe689\") " Jul 07 00:07:39.571521 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:07:39.571357 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjvph\" (UniqueName: \"kubernetes.io/projected/92e2d5ca-67d5-46e1-bd03-d236964fe689-kube-api-access-tjvph\") pod \"92e2d5ca-67d5-46e1-bd03-d236964fe689\" (UID: \"92e2d5ca-67d5-46e1-bd03-d236964fe689\") " Jul 07 00:07:39.571782 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:07:39.571750 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92e2d5ca-67d5-46e1-bd03-d236964fe689-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "92e2d5ca-67d5-46e1-bd03-d236964fe689" (UID: "92e2d5ca-67d5-46e1-bd03-d236964fe689"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:07:39.571933 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:07:39.571901 2553 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/92e2d5ca-67d5-46e1-bd03-d236964fe689-extensions-socket-volume\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:07:39.573515 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:07:39.573491 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92e2d5ca-67d5-46e1-bd03-d236964fe689-kube-api-access-tjvph" (OuterVolumeSpecName: "kube-api-access-tjvph") pod "92e2d5ca-67d5-46e1-bd03-d236964fe689" (UID: "92e2d5ca-67d5-46e1-bd03-d236964fe689"). InnerVolumeSpecName "kube-api-access-tjvph". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:07:39.673019 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:07:39.672986 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjvph\" (UniqueName: \"kubernetes.io/projected/92e2d5ca-67d5-46e1-bd03-d236964fe689-kube-api-access-tjvph\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:07:39.869563 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:07:39.869393 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7nh2l"] Jul 07 00:07:39.870974 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:07:39.870946 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7nh2l"] Jul 07 00:07:40.303378 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:07:40.303347 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92e2d5ca-67d5-46e1-bd03-d236964fe689" path="/var/lib/kubelet/pods/92e2d5ca-67d5-46e1-bd03-d236964fe689/volumes" Jul 07 00:08:36.559182 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:08:36.559140 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7pmgv"] Jul 07 00:08:36.559673 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:08:36.559586 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92e2d5ca-67d5-46e1-bd03-d236964fe689" containerName="manager" Jul 07 00:08:36.559673 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:08:36.559598 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="92e2d5ca-67d5-46e1-bd03-d236964fe689" containerName="manager" Jul 07 00:08:36.559754 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:08:36.559677 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="92e2d5ca-67d5-46e1-bd03-d236964fe689" containerName="manager" Jul 07 00:08:36.562850 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:08:36.562832 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7pmgv" Jul 07 00:08:36.565705 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:08:36.565683 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hrvd6\"" Jul 07 00:08:36.574518 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:08:36.574491 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7pmgv"] Jul 07 00:08:36.702844 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:08:36.702797 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf17cf71-15c2-40b4-81bb-df10ca03011c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7pmgv\" (UID: \"bf17cf71-15c2-40b4-81bb-df10ca03011c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7pmgv" Jul 07 00:08:36.703035 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:08:36.702924 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl6sp\" (UniqueName: \"kubernetes.io/projected/bf17cf71-15c2-40b4-81bb-df10ca03011c-kube-api-access-pl6sp\") pod \"kuadrant-operator-controller-manager-86878c769f-7pmgv\" (UID: \"bf17cf71-15c2-40b4-81bb-df10ca03011c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7pmgv" Jul 07 00:08:36.804492 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:08:36.804441 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf17cf71-15c2-40b4-81bb-df10ca03011c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7pmgv\" (UID: \"bf17cf71-15c2-40b4-81bb-df10ca03011c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7pmgv" Jul 07 00:08:36.804692 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:08:36.804545 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl6sp\" (UniqueName: \"kubernetes.io/projected/bf17cf71-15c2-40b4-81bb-df10ca03011c-kube-api-access-pl6sp\") pod \"kuadrant-operator-controller-manager-86878c769f-7pmgv\" (UID: \"bf17cf71-15c2-40b4-81bb-df10ca03011c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7pmgv" Jul 07 00:08:36.804867 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:08:36.804844 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf17cf71-15c2-40b4-81bb-df10ca03011c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7pmgv\" (UID: \"bf17cf71-15c2-40b4-81bb-df10ca03011c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7pmgv" Jul 07 00:08:36.814776 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:08:36.814720 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl6sp\" (UniqueName: \"kubernetes.io/projected/bf17cf71-15c2-40b4-81bb-df10ca03011c-kube-api-access-pl6sp\") pod \"kuadrant-operator-controller-manager-86878c769f-7pmgv\" (UID: \"bf17cf71-15c2-40b4-81bb-df10ca03011c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7pmgv" Jul 07 00:08:36.874118 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:08:36.874073 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7pmgv" Jul 07 00:08:37.033910 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:08:37.033885 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7pmgv"] Jul 07 00:08:37.034857 ip-10-0-136-25 kubenswrapper[2553]: W0707 00:08:37.034827 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf17cf71_15c2_40b4_81bb_df10ca03011c.slice/crio-99268e8d899d4c12346671ca8e11927a89212bab23e4841d54b45137cfa7d063 WatchSource:0}: Error finding container 99268e8d899d4c12346671ca8e11927a89212bab23e4841d54b45137cfa7d063: Status 404 returned error can't find the container with id 99268e8d899d4c12346671ca8e11927a89212bab23e4841d54b45137cfa7d063 Jul 07 00:08:37.873923 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:08:37.873885 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7pmgv" event={"ID":"bf17cf71-15c2-40b4-81bb-df10ca03011c","Type":"ContainerStarted","Data":"cb2031125405398c0c3b0e92807efdab0c2ddc141d4a588f1b8e7739eff02650"} Jul 07 00:08:37.874358 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:08:37.873938 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7pmgv" event={"ID":"bf17cf71-15c2-40b4-81bb-df10ca03011c","Type":"ContainerStarted","Data":"99268e8d899d4c12346671ca8e11927a89212bab23e4841d54b45137cfa7d063"} Jul 07 00:08:37.874358 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:08:37.873968 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7pmgv" Jul 07 00:08:37.919421 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:08:37.919363 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7pmgv" podStartSLOduration=1.919345404 podStartE2EDuration="1.919345404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:08:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:08:37.918985015 +0000 UTC m=+1200.128714613" watchObservedRunningTime="2026-07-07 00:08:37.919345404 +0000 UTC m=+1200.129074925" Jul 07 00:08:38.288882 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:08:38.288842 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4z8vq_3253bdfe-5248-4d4c-9864-11660eb042ca/ovn-acl-logging/0.log" Jul 07 00:08:38.291386 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:08:38.291363 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4z8vq_3253bdfe-5248-4d4c-9864-11660eb042ca/ovn-acl-logging/0.log" Jul 07 00:08:48.882483 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:08:48.882449 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7pmgv" Jul 07 00:13:38.332532 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:13:38.332498 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4z8vq_3253bdfe-5248-4d4c-9864-11660eb042ca/ovn-acl-logging/0.log" Jul 07 00:13:38.335055 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:13:38.332629 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4z8vq_3253bdfe-5248-4d4c-9864-11660eb042ca/ovn-acl-logging/0.log" Jul 07 00:14:56.285811 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.285767 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk"] Jul 07 00:14:56.289824 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.289623 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:56.292323 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.292291 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-aab879b8-openshift-default-dockercfg-fs68h\"" Jul 07 00:14:56.310363 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.310339 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk"] Jul 07 00:14:56.364617 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.364589 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-credential-socket\") pod \"e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:56.364791 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.364626 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-workload-socket\") pod \"e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:56.364791 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.364653 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istio-token\") pod \"e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:56.364791 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.364683 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-workload-certs\") pod \"e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:56.364791 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.364712 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istio-envoy\") pod \"e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:56.364791 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.364737 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istio-data\") pod \"e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:56.365037 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.364805 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istio-podinfo\") pod \"e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:56.365037 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.364863 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istiod-ca-cert\") pod \"e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:56.365037 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.364889 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tj9b\" (UniqueName: \"kubernetes.io/projected/4988826d-1588-4eee-b4b1-8eba2cda5ee1-kube-api-access-7tj9b\") pod \"e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:56.465614 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.465580 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-credential-socket\") pod \"e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:56.465614 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.465619 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-workload-socket\") pod \"e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:56.465878 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.465643 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istio-token\") pod \"e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:56.465878 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.465660 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-workload-certs\") pod \"e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:56.465878 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.465675 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istio-envoy\") pod \"e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:56.465878 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.465692 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istio-data\") pod \"e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:56.465878 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.465722 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istio-podinfo\") pod \"e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:56.465878 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.465760 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istiod-ca-cert\") pod \"e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:56.465878 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.465785 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tj9b\" (UniqueName: \"kubernetes.io/projected/4988826d-1588-4eee-b4b1-8eba2cda5ee1-kube-api-access-7tj9b\") pod \"e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:56.466199 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.466055 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-workload-certs\") pod \"e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:56.466199 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.466081 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-workload-socket\") pod \"e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:56.466272 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.466204 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istio-data\") pod \"e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:56.466367 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.466347 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-credential-socket\") pod \"e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:56.466611 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.466590 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istiod-ca-cert\") pod \"e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:56.468110 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.468075 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istio-envoy\") pod \"e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:56.468341 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.468322 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istio-podinfo\") pod \"e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:56.476386 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.476363 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istio-token\") pod \"e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:56.477255 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.477231 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tj9b\" (UniqueName: \"kubernetes.io/projected/4988826d-1588-4eee-b4b1-8eba2cda5ee1-kube-api-access-7tj9b\") pod \"e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:56.604613 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.604537 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:56.749681 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.749654 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk"] Jul 07 00:14:56.750601 ip-10-0-136-25 kubenswrapper[2553]: W0707 00:14:56.750578 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4988826d_1588_4eee_b4b1_8eba2cda5ee1.slice/crio-a6faff20ec4cfab6e0e3e12d2da2f3adb167a72aea07500453b529ef4a48d77f WatchSource:0}: Error finding container a6faff20ec4cfab6e0e3e12d2da2f3adb167a72aea07500453b529ef4a48d77f: Status 404 returned error can't find the container with id a6faff20ec4cfab6e0e3e12d2da2f3adb167a72aea07500453b529ef4a48d77f Jul 07 00:14:56.753127 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.753097 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:14:56.753229 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.753175 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:14:56.753229 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:56.753219 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:14:57.116232 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:57.116194 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" event={"ID":"4988826d-1588-4eee-b4b1-8eba2cda5ee1","Type":"ContainerStarted","Data":"cdf93efe2a3f60beabc419530f0923b39b2ad23d5f091ceba334c07dd211ce49"} Jul 07 00:14:57.116442 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:57.116256 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" event={"ID":"4988826d-1588-4eee-b4b1-8eba2cda5ee1","Type":"ContainerStarted","Data":"a6faff20ec4cfab6e0e3e12d2da2f3adb167a72aea07500453b529ef4a48d77f"} Jul 07 00:14:57.165842 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:57.165788 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" podStartSLOduration=1.165772577 podStartE2EDuration="1.165772577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:14:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:14:57.16502378 +0000 UTC m=+1579.374753328" watchObservedRunningTime="2026-07-07 00:14:57.165772577 +0000 UTC m=+1579.375502097" Jul 07 00:14:57.604766 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:57.604730 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:57.609741 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:57.609713 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:58.123253 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:58.123221 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:14:58.124081 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:14:58.124063 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:15:02.757698 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:02.757660 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk"] Jul 07 00:15:02.758281 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:02.757882 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" podUID="4988826d-1588-4eee-b4b1-8eba2cda5ee1" containerName="istio-proxy" containerID="cri-o://cdf93efe2a3f60beabc419530f0923b39b2ad23d5f091ceba334c07dd211ce49" gracePeriod=30 Jul 07 00:15:08.437378 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.437343 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:15:08.594374 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.594248 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-workload-certs\") pod \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " Jul 07 00:15:08.594374 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.594342 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istio-data\") pod \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " Jul 07 00:15:08.594374 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.594365 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-credential-socket\") pod \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " Jul 07 00:15:08.594652 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.594384 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istio-envoy\") pod \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " Jul 07 00:15:08.594652 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.594439 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-workload-socket\") pod \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " Jul 07 00:15:08.594652 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.594467 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istiod-ca-cert\") pod \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " Jul 07 00:15:08.594652 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.594499 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7tj9b\" (UniqueName: \"kubernetes.io/projected/4988826d-1588-4eee-b4b1-8eba2cda5ee1-kube-api-access-7tj9b\") pod \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " Jul 07 00:15:08.594652 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.594534 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istio-token\") pod \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " Jul 07 00:15:08.594652 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.594534 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4988826d-1588-4eee-b4b1-8eba2cda5ee1" (UID: "4988826d-1588-4eee-b4b1-8eba2cda5ee1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:15:08.594652 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.594574 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istio-podinfo\") pod \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\" (UID: \"4988826d-1588-4eee-b4b1-8eba2cda5ee1\") " Jul 07 00:15:08.595028 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.594656 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4988826d-1588-4eee-b4b1-8eba2cda5ee1" (UID: "4988826d-1588-4eee-b4b1-8eba2cda5ee1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:15:08.595028 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.594764 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4988826d-1588-4eee-b4b1-8eba2cda5ee1" (UID: "4988826d-1588-4eee-b4b1-8eba2cda5ee1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:15:08.595028 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.594925 2553 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-workload-socket\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:15:08.595028 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.594945 2553 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-workload-certs\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:15:08.595028 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.594958 2553 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-credential-socket\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:15:08.595028 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.595020 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4988826d-1588-4eee-b4b1-8eba2cda5ee1" (UID: "4988826d-1588-4eee-b4b1-8eba2cda5ee1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:15:08.595369 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.595203 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istio-data" (OuterVolumeSpecName: "istio-data") pod "4988826d-1588-4eee-b4b1-8eba2cda5ee1" (UID: "4988826d-1588-4eee-b4b1-8eba2cda5ee1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:15:08.596725 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.596693 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4988826d-1588-4eee-b4b1-8eba2cda5ee1" (UID: "4988826d-1588-4eee-b4b1-8eba2cda5ee1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:15:08.596725 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.596714 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istio-token" (OuterVolumeSpecName: "istio-token") pod "4988826d-1588-4eee-b4b1-8eba2cda5ee1" (UID: "4988826d-1588-4eee-b4b1-8eba2cda5ee1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:15:08.596898 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.596728 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4988826d-1588-4eee-b4b1-8eba2cda5ee1" (UID: "4988826d-1588-4eee-b4b1-8eba2cda5ee1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 00:15:08.596898 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.596775 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4988826d-1588-4eee-b4b1-8eba2cda5ee1-kube-api-access-7tj9b" (OuterVolumeSpecName: "kube-api-access-7tj9b") pod "4988826d-1588-4eee-b4b1-8eba2cda5ee1" (UID: "4988826d-1588-4eee-b4b1-8eba2cda5ee1"). InnerVolumeSpecName "kube-api-access-7tj9b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:15:08.696316 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.696261 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tj9b\" (UniqueName: \"kubernetes.io/projected/4988826d-1588-4eee-b4b1-8eba2cda5ee1-kube-api-access-7tj9b\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:15:08.696316 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.696324 2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istio-token\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:15:08.696539 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.696335 2553 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istio-podinfo\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:15:08.696539 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.696343 2553 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istio-data\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:15:08.696539 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.696351 2553 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istio-envoy\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:15:08.696539 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.696360 2553 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4988826d-1588-4eee-b4b1-8eba2cda5ee1-istiod-ca-cert\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:15:08.947487 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.947451 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb"] Jul 07 00:15:08.948074 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.948048 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4988826d-1588-4eee-b4b1-8eba2cda5ee1" containerName="istio-proxy" Jul 07 00:15:08.948074 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.948075 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="4988826d-1588-4eee-b4b1-8eba2cda5ee1" containerName="istio-proxy" Jul 07 00:15:08.948278 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.948208 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="4988826d-1588-4eee-b4b1-8eba2cda5ee1" containerName="istio-proxy" Jul 07 00:15:08.951556 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.951533 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:08.954505 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.954477 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-4c7f79fd-openshift-default-dockercfg-mtsms\"" Jul 07 00:15:08.964429 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:08.964401 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb"] Jul 07 00:15:09.099983 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.099951 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-workload-socket\") pod \"e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:09.099983 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.099986 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istio-envoy\") pod \"e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:09.100255 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.100006 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istiod-ca-cert\") pod \"e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:09.100255 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.100036 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qb59m\" (UniqueName: \"kubernetes.io/projected/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-kube-api-access-qb59m\") pod \"e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:09.100255 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.100092 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-credential-socket\") pod \"e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:09.100255 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.100126 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istio-data\") pod \"e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:09.100255 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.100146 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-workload-certs\") pod \"e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:09.100255 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.100167 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istio-podinfo\") pod \"e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:09.100255 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.100183 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istio-token\") pod \"e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:09.193193 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.193152 2553 generic.go:356] "Generic (PLEG): container finished" podID="4988826d-1588-4eee-b4b1-8eba2cda5ee1" containerID="cdf93efe2a3f60beabc419530f0923b39b2ad23d5f091ceba334c07dd211ce49" exitCode=0 Jul 07 00:15:09.193439 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.193203 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" Jul 07 00:15:09.193439 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.193235 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" event={"ID":"4988826d-1588-4eee-b4b1-8eba2cda5ee1","Type":"ContainerDied","Data":"cdf93efe2a3f60beabc419530f0923b39b2ad23d5f091ceba334c07dd211ce49"} Jul 07 00:15:09.193439 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.193286 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk" event={"ID":"4988826d-1588-4eee-b4b1-8eba2cda5ee1","Type":"ContainerDied","Data":"a6faff20ec4cfab6e0e3e12d2da2f3adb167a72aea07500453b529ef4a48d77f"} Jul 07 00:15:09.193439 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.193330 2553 scope.go:117] "RemoveContainer" containerID="cdf93efe2a3f60beabc419530f0923b39b2ad23d5f091ceba334c07dd211ce49" Jul 07 00:15:09.201257 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.200823 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-workload-socket\") pod \"e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:09.201257 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.200851 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istio-envoy\") pod \"e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:09.201257 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.200884 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istiod-ca-cert\") pod \"e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:09.201257 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.200924 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb59m\" (UniqueName: \"kubernetes.io/projected/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-kube-api-access-qb59m\") pod \"e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:09.201257 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.200962 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-credential-socket\") pod \"e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:09.201257 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.200993 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istio-data\") pod \"e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:09.201257 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.201027 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-workload-certs\") pod \"e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:09.201257 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.201057 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istio-podinfo\") pod \"e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:09.201257 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.201077 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istio-token\") pod \"e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:09.201837 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.201533 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-credential-socket\") pod \"e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:09.201837 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.201720 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-workload-socket\") pod \"e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:09.201998 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.201973 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istio-data\") pod \"e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:09.202163 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.202145 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istiod-ca-cert\") pod \"e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:09.202247 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.202219 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-workload-certs\") pod \"e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:09.203592 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.203484 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istio-envoy\") pod \"e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:09.203663 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.203599 2553 scope.go:117] "RemoveContainer" containerID="cdf93efe2a3f60beabc419530f0923b39b2ad23d5f091ceba334c07dd211ce49" Jul 07 00:15:09.203899 ip-10-0-136-25 kubenswrapper[2553]: E0707 00:15:09.203881 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdf93efe2a3f60beabc419530f0923b39b2ad23d5f091ceba334c07dd211ce49\": container with ID starting with cdf93efe2a3f60beabc419530f0923b39b2ad23d5f091ceba334c07dd211ce49 not found: ID does not exist" containerID="cdf93efe2a3f60beabc419530f0923b39b2ad23d5f091ceba334c07dd211ce49" Jul 07 00:15:09.203941 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.203908 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdf93efe2a3f60beabc419530f0923b39b2ad23d5f091ceba334c07dd211ce49"} err="failed to get container status \"cdf93efe2a3f60beabc419530f0923b39b2ad23d5f091ceba334c07dd211ce49\": rpc error: code = NotFound desc = could not find container \"cdf93efe2a3f60beabc419530f0923b39b2ad23d5f091ceba334c07dd211ce49\": container with ID starting with cdf93efe2a3f60beabc419530f0923b39b2ad23d5f091ceba334c07dd211ce49 not found: ID does not exist" Jul 07 00:15:09.203941 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.203883 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istio-podinfo\") pod \"e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:09.209725 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.209698 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qb59m\" (UniqueName: \"kubernetes.io/projected/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-kube-api-access-qb59m\") pod \"e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:09.209830 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.209792 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istio-token\") pod \"e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:09.220605 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.220579 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk"] Jul 07 00:15:09.226373 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.226345 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-aab879b8-openshift-default-84cf9d4f5-2fltk"] Jul 07 00:15:09.266829 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.266791 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:09.427767 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.427740 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb"] Jul 07 00:15:09.430113 ip-10-0-136-25 kubenswrapper[2553]: W0707 00:15:09.430086 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c3c0721_f35f_4dbb_8eb0_fbc9253080b9.slice/crio-8f26666f420435585749c79c06271471e277c902e09d824488c46016f0263b69 WatchSource:0}: Error finding container 8f26666f420435585749c79c06271471e277c902e09d824488c46016f0263b69: Status 404 returned error can't find the container with id 8f26666f420435585749c79c06271471e277c902e09d824488c46016f0263b69 Jul 07 00:15:09.432424 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.432389 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:15:09.432537 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.432459 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:15:09.432537 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:09.432494 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:15:10.213565 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:10.213525 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" event={"ID":"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9","Type":"ContainerStarted","Data":"4b9ea9cf2b919616315e597851b7e3fe58e1ce3e7b6e69ad92ae27a109463774"} Jul 07 00:15:10.214053 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:10.213609 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" event={"ID":"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9","Type":"ContainerStarted","Data":"8f26666f420435585749c79c06271471e277c902e09d824488c46016f0263b69"} Jul 07 00:15:10.236713 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:10.236664 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" podStartSLOduration=2.236647525 podStartE2EDuration="2.236647525s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:15:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:15:10.235248739 +0000 UTC m=+1592.444978261" watchObservedRunningTime="2026-07-07 00:15:10.236647525 +0000 UTC m=+1592.446377046" Jul 07 00:15:10.267140 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:10.267106 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:10.268691 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:10.268666 2553 patch_prober.go:28] interesting pod/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Jul 07 00:15:10.268775 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:10.268723 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" podUID="1c3c0721-f35f-4dbb-8eb0-fbc9253080b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Jul 07 00:15:10.303858 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:10.303823 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4988826d-1588-4eee-b4b1-8eba2cda5ee1" path="/var/lib/kubelet/pods/4988826d-1588-4eee-b4b1-8eba2cda5ee1/volumes" Jul 07 00:15:11.270798 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:11.270770 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:12.225630 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:12.225597 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:12.226746 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:12.226726 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:21.291623 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:21.291532 2553 generic.go:356] "Generic (PLEG): container finished" podID="1c3c0721-f35f-4dbb-8eb0-fbc9253080b9" containerID="4b9ea9cf2b919616315e597851b7e3fe58e1ce3e7b6e69ad92ae27a109463774" exitCode=0 Jul 07 00:15:21.291623 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:21.291595 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" event={"ID":"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9","Type":"ContainerDied","Data":"4b9ea9cf2b919616315e597851b7e3fe58e1ce3e7b6e69ad92ae27a109463774"} Jul 07 00:15:21.292150 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:21.292007 2553 scope.go:117] "RemoveContainer" containerID="4b9ea9cf2b919616315e597851b7e3fe58e1ce3e7b6e69ad92ae27a109463774" Jul 07 00:15:21.293042 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:21.293012 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:15:21.293153 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:21.293094 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:15:21.293218 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:21.293169 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:15:21.395371 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:21.395331 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb"] Jul 07 00:15:22.299943 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:22.299907 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" podUID="1c3c0721-f35f-4dbb-8eb0-fbc9253080b9" containerName="istio-proxy" containerID="cri-o://641a6d867fa9a72d2ba7ce19a367f76d4b964d17848cbb5eb3932018a7952e30" gracePeriod=30 Jul 07 00:15:22.304997 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:22.304966 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" event={"ID":"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9","Type":"ContainerStarted","Data":"641a6d867fa9a72d2ba7ce19a367f76d4b964d17848cbb5eb3932018a7952e30"} Jul 07 00:15:27.226205 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.226162 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:27.572914 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.572888 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:27.684944 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.684905 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-workload-socket\") pod \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " Jul 07 00:15:27.685133 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.684967 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istio-token\") pod \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " Jul 07 00:15:27.685133 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.684995 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istio-envoy\") pod \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " Jul 07 00:15:27.685133 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.685015 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istio-podinfo\") pod \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " Jul 07 00:15:27.685133 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.685057 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istiod-ca-cert\") pod \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " Jul 07 00:15:27.685405 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.685146 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qb59m\" (UniqueName: \"kubernetes.io/projected/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-kube-api-access-qb59m\") pod \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " Jul 07 00:15:27.685405 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.685191 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istio-data\") pod \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " Jul 07 00:15:27.685405 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.685221 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-credential-socket\") pod \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " Jul 07 00:15:27.685405 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.685250 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-workload-certs\") pod \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\" (UID: \"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9\") " Jul 07 00:15:27.685646 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.685212 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1c3c0721-f35f-4dbb-8eb0-fbc9253080b9" (UID: "1c3c0721-f35f-4dbb-8eb0-fbc9253080b9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:15:27.685708 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.685672 2553 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-workload-socket\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:15:27.685708 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.685462 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1c3c0721-f35f-4dbb-8eb0-fbc9253080b9" (UID: "1c3c0721-f35f-4dbb-8eb0-fbc9253080b9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:15:27.685708 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.685565 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1c3c0721-f35f-4dbb-8eb0-fbc9253080b9" (UID: "1c3c0721-f35f-4dbb-8eb0-fbc9253080b9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:15:27.685857 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.685774 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1c3c0721-f35f-4dbb-8eb0-fbc9253080b9" (UID: "1c3c0721-f35f-4dbb-8eb0-fbc9253080b9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:15:27.686241 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.686214 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istio-data" (OuterVolumeSpecName: "istio-data") pod "1c3c0721-f35f-4dbb-8eb0-fbc9253080b9" (UID: "1c3c0721-f35f-4dbb-8eb0-fbc9253080b9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:15:27.687430 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.687404 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1c3c0721-f35f-4dbb-8eb0-fbc9253080b9" (UID: "1c3c0721-f35f-4dbb-8eb0-fbc9253080b9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:15:27.687532 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.687442 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istio-token" (OuterVolumeSpecName: "istio-token") pod "1c3c0721-f35f-4dbb-8eb0-fbc9253080b9" (UID: "1c3c0721-f35f-4dbb-8eb0-fbc9253080b9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:15:27.687704 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.687669 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-kube-api-access-qb59m" (OuterVolumeSpecName: "kube-api-access-qb59m") pod "1c3c0721-f35f-4dbb-8eb0-fbc9253080b9" (UID: "1c3c0721-f35f-4dbb-8eb0-fbc9253080b9"). InnerVolumeSpecName "kube-api-access-qb59m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:15:27.687704 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.687680 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1c3c0721-f35f-4dbb-8eb0-fbc9253080b9" (UID: "1c3c0721-f35f-4dbb-8eb0-fbc9253080b9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 00:15:27.787056 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.787022 2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istio-token\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:15:27.787056 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.787051 2553 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istio-envoy\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:15:27.787056 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.787060 2553 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istio-podinfo\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:15:27.787056 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.787067 2553 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istiod-ca-cert\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:15:27.787344 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.787076 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qb59m\" (UniqueName: \"kubernetes.io/projected/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-kube-api-access-qb59m\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:15:27.787344 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.787085 2553 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-istio-data\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:15:27.787344 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.787094 2553 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-credential-socket\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:15:27.787344 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.787103 2553 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9-workload-certs\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:15:27.890225 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.890191 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg"] Jul 07 00:15:27.890877 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.890852 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c3c0721-f35f-4dbb-8eb0-fbc9253080b9" containerName="istio-proxy" Jul 07 00:15:27.890877 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.890878 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c3c0721-f35f-4dbb-8eb0-fbc9253080b9" containerName="istio-proxy" Jul 07 00:15:27.891058 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.890897 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c3c0721-f35f-4dbb-8eb0-fbc9253080b9" containerName="istio-proxy" Jul 07 00:15:27.891058 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.890907 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c3c0721-f35f-4dbb-8eb0-fbc9253080b9" containerName="istio-proxy" Jul 07 00:15:27.891058 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.891038 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c3c0721-f35f-4dbb-8eb0-fbc9253080b9" containerName="istio-proxy" Jul 07 00:15:27.894926 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.894904 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:27.899911 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.899886 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-2a6aa737-openshift-default-dockercfg-mvc2l\"" Jul 07 00:15:27.910002 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.909974 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg"] Jul 07 00:15:27.988490 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.988457 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-workload-socket\") pod \"e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:27.988669 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.988498 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istio-podinfo\") pod \"e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:27.988669 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.988544 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-credential-socket\") pod \"e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:27.988669 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.988568 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istio-envoy\") pod \"e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:27.988669 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.988584 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-workload-certs\") pod \"e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:27.988669 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.988604 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istio-data\") pod \"e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:27.988884 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.988681 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcdnz\" (UniqueName: \"kubernetes.io/projected/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-kube-api-access-rcdnz\") pod \"e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:27.988884 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.988726 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istiod-ca-cert\") pod \"e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:27.988884 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:27.988787 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istio-token\") pod \"e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:28.090209 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.090147 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-workload-socket\") pod \"e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:28.090209 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.090218 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istio-podinfo\") pod \"e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:28.090475 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.090365 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-credential-socket\") pod \"e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:28.090475 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.090419 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istio-envoy\") pod \"e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:28.090475 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.090445 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-workload-certs\") pod \"e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:28.090587 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.090485 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istio-data\") pod \"e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:28.090673 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.090648 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcdnz\" (UniqueName: \"kubernetes.io/projected/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-kube-api-access-rcdnz\") pod \"e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:28.090737 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.090703 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-workload-socket\") pod \"e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:28.090737 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.090711 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istiod-ca-cert\") pod \"e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:28.090838 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.090754 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-credential-socket\") pod \"e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:28.090838 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.090795 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istio-data\") pod \"e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:28.090838 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.090826 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istio-token\") pod \"e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:28.091002 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.090905 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-workload-certs\") pod \"e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:28.091415 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.091394 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istiod-ca-cert\") pod \"e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:28.093166 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.093150 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istio-envoy\") pod \"e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:28.093398 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.093380 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istio-podinfo\") pod \"e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:28.114818 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.114789 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istio-token\") pod \"e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:28.115207 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.115190 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcdnz\" (UniqueName: \"kubernetes.io/projected/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-kube-api-access-rcdnz\") pod \"e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:28.206429 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.206394 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:28.339798 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.339768 2553 generic.go:356] "Generic (PLEG): container finished" podID="1c3c0721-f35f-4dbb-8eb0-fbc9253080b9" containerID="641a6d867fa9a72d2ba7ce19a367f76d4b964d17848cbb5eb3932018a7952e30" exitCode=0 Jul 07 00:15:28.340157 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.339817 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" Jul 07 00:15:28.340157 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.339835 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" event={"ID":"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9","Type":"ContainerDied","Data":"641a6d867fa9a72d2ba7ce19a367f76d4b964d17848cbb5eb3932018a7952e30"} Jul 07 00:15:28.340157 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.339876 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb" event={"ID":"1c3c0721-f35f-4dbb-8eb0-fbc9253080b9","Type":"ContainerDied","Data":"8f26666f420435585749c79c06271471e277c902e09d824488c46016f0263b69"} Jul 07 00:15:28.340157 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.339896 2553 scope.go:117] "RemoveContainer" containerID="641a6d867fa9a72d2ba7ce19a367f76d4b964d17848cbb5eb3932018a7952e30" Jul 07 00:15:28.349316 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.349285 2553 scope.go:117] "RemoveContainer" containerID="4b9ea9cf2b919616315e597851b7e3fe58e1ce3e7b6e69ad92ae27a109463774" Jul 07 00:15:28.357213 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.357195 2553 scope.go:117] "RemoveContainer" containerID="641a6d867fa9a72d2ba7ce19a367f76d4b964d17848cbb5eb3932018a7952e30" Jul 07 00:15:28.357486 ip-10-0-136-25 kubenswrapper[2553]: E0707 00:15:28.357461 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"641a6d867fa9a72d2ba7ce19a367f76d4b964d17848cbb5eb3932018a7952e30\": container with ID starting with 641a6d867fa9a72d2ba7ce19a367f76d4b964d17848cbb5eb3932018a7952e30 not found: ID does not exist" containerID="641a6d867fa9a72d2ba7ce19a367f76d4b964d17848cbb5eb3932018a7952e30" Jul 07 00:15:28.357576 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.357544 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"641a6d867fa9a72d2ba7ce19a367f76d4b964d17848cbb5eb3932018a7952e30"} err="failed to get container status \"641a6d867fa9a72d2ba7ce19a367f76d4b964d17848cbb5eb3932018a7952e30\": rpc error: code = NotFound desc = could not find container \"641a6d867fa9a72d2ba7ce19a367f76d4b964d17848cbb5eb3932018a7952e30\": container with ID starting with 641a6d867fa9a72d2ba7ce19a367f76d4b964d17848cbb5eb3932018a7952e30 not found: ID does not exist" Jul 07 00:15:28.357576 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.357562 2553 scope.go:117] "RemoveContainer" containerID="4b9ea9cf2b919616315e597851b7e3fe58e1ce3e7b6e69ad92ae27a109463774" Jul 07 00:15:28.357911 ip-10-0-136-25 kubenswrapper[2553]: E0707 00:15:28.357821 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b9ea9cf2b919616315e597851b7e3fe58e1ce3e7b6e69ad92ae27a109463774\": container with ID starting with 4b9ea9cf2b919616315e597851b7e3fe58e1ce3e7b6e69ad92ae27a109463774 not found: ID does not exist" containerID="4b9ea9cf2b919616315e597851b7e3fe58e1ce3e7b6e69ad92ae27a109463774" Jul 07 00:15:28.357911 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.357890 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b9ea9cf2b919616315e597851b7e3fe58e1ce3e7b6e69ad92ae27a109463774"} err="failed to get container status \"4b9ea9cf2b919616315e597851b7e3fe58e1ce3e7b6e69ad92ae27a109463774\": rpc error: code = NotFound desc = could not find container \"4b9ea9cf2b919616315e597851b7e3fe58e1ce3e7b6e69ad92ae27a109463774\": container with ID starting with 4b9ea9cf2b919616315e597851b7e3fe58e1ce3e7b6e69ad92ae27a109463774 not found: ID does not exist" Jul 07 00:15:28.361310 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.361277 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg"] Jul 07 00:15:28.362678 ip-10-0-136-25 kubenswrapper[2553]: W0707 00:15:28.362622 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9738ebe4_6ca2_48bf_8cfb_41d12bfd18ed.slice/crio-54478568cdc8f1c35df3833a3b6ef00a2aa18b87fceec33568dcf6ec664419a0 WatchSource:0}: Error finding container 54478568cdc8f1c35df3833a3b6ef00a2aa18b87fceec33568dcf6ec664419a0: Status 404 returned error can't find the container with id 54478568cdc8f1c35df3833a3b6ef00a2aa18b87fceec33568dcf6ec664419a0 Jul 07 00:15:28.365093 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.365067 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:15:28.365166 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.365128 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:15:28.365166 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.365159 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:15:28.366433 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.366413 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb"] Jul 07 00:15:28.373411 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:28.373387 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-4c7f79fd-openshift-default-7c75dbc7f-lplcb"] Jul 07 00:15:29.348126 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:29.348086 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" event={"ID":"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed","Type":"ContainerStarted","Data":"6f6fb5e208cd371ce176a8ec4882a197fd8a7669e041b2c6a5eae598f454508d"} Jul 07 00:15:29.348549 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:29.348146 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" event={"ID":"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed","Type":"ContainerStarted","Data":"54478568cdc8f1c35df3833a3b6ef00a2aa18b87fceec33568dcf6ec664419a0"} Jul 07 00:15:29.370887 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:29.370836 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" podStartSLOduration=2.370820595 podStartE2EDuration="2.370820595s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:15:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:15:29.368877169 +0000 UTC m=+1611.578606690" watchObservedRunningTime="2026-07-07 00:15:29.370820595 +0000 UTC m=+1611.580550115" Jul 07 00:15:30.206681 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:30.206595 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:30.212252 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:30.212216 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:30.304062 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:30.304023 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c3c0721-f35f-4dbb-8eb0-fbc9253080b9" path="/var/lib/kubelet/pods/1c3c0721-f35f-4dbb-8eb0-fbc9253080b9/volumes" Jul 07 00:15:30.354031 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:30.353998 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:30.354933 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:30.354915 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:33.677439 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:33.677404 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg"] Jul 07 00:15:33.677832 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:33.677585 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" podUID="9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed" containerName="istio-proxy" containerID="cri-o://6f6fb5e208cd371ce176a8ec4882a197fd8a7669e041b2c6a5eae598f454508d" gracePeriod=30 Jul 07 00:15:38.948900 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:38.948834 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:39.002635 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.002600 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-workload-socket\") pod \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " Jul 07 00:15:39.002806 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.002659 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istio-envoy\") pod \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " Jul 07 00:15:39.002806 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.002678 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rcdnz\" (UniqueName: \"kubernetes.io/projected/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-kube-api-access-rcdnz\") pod \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " Jul 07 00:15:39.002806 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.002699 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istio-data\") pod \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " Jul 07 00:15:39.002806 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.002762 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istio-podinfo\") pod \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " Jul 07 00:15:39.002806 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.002784 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istiod-ca-cert\") pod \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " Jul 07 00:15:39.003070 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.002835 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-workload-certs\") pod \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " Jul 07 00:15:39.003070 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.002865 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-credential-socket\") pod \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " Jul 07 00:15:39.003070 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.002892 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istio-token\") pod \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\" (UID: \"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed\") " Jul 07 00:15:39.003070 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.003015 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istio-data" (OuterVolumeSpecName: "istio-data") pod "9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed" (UID: "9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:15:39.003278 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.003067 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed" (UID: "9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:15:39.003278 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.003185 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed" (UID: "9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:15:39.003419 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.003314 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed" (UID: "9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:15:39.003419 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.003335 2553 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istio-data\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:15:39.003419 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.003357 2553 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istiod-ca-cert\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:15:39.003419 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.003370 2553 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-workload-certs\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:15:39.003419 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.003388 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed" (UID: "9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:15:39.004983 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.004949 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-kube-api-access-rcdnz" (OuterVolumeSpecName: "kube-api-access-rcdnz") pod "9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed" (UID: "9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed"). InnerVolumeSpecName "kube-api-access-rcdnz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:15:39.005117 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.005032 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed" (UID: "9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:15:39.005117 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.005043 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed" (UID: "9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 00:15:39.005117 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.005066 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istio-token" (OuterVolumeSpecName: "istio-token") pod "9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed" (UID: "9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:15:39.104377 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.104340 2553 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-workload-socket\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:15:39.104377 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.104372 2553 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istio-envoy\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:15:39.104377 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.104382 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcdnz\" (UniqueName: \"kubernetes.io/projected/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-kube-api-access-rcdnz\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:15:39.104377 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.104392 2553 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istio-podinfo\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:15:39.104656 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.104400 2553 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-credential-socket\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:15:39.104656 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.104408 2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed-istio-token\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:15:39.414469 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.414435 2553 generic.go:356] "Generic (PLEG): container finished" podID="9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed" containerID="6f6fb5e208cd371ce176a8ec4882a197fd8a7669e041b2c6a5eae598f454508d" exitCode=0 Jul 07 00:15:39.414648 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.414473 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" Jul 07 00:15:39.414648 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.414510 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" event={"ID":"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed","Type":"ContainerDied","Data":"6f6fb5e208cd371ce176a8ec4882a197fd8a7669e041b2c6a5eae598f454508d"} Jul 07 00:15:39.414648 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.414566 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg" event={"ID":"9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed","Type":"ContainerDied","Data":"54478568cdc8f1c35df3833a3b6ef00a2aa18b87fceec33568dcf6ec664419a0"} Jul 07 00:15:39.414648 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.414591 2553 scope.go:117] "RemoveContainer" containerID="6f6fb5e208cd371ce176a8ec4882a197fd8a7669e041b2c6a5eae598f454508d" Jul 07 00:15:39.426740 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.426561 2553 scope.go:117] "RemoveContainer" containerID="6f6fb5e208cd371ce176a8ec4882a197fd8a7669e041b2c6a5eae598f454508d" Jul 07 00:15:39.426874 ip-10-0-136-25 kubenswrapper[2553]: E0707 00:15:39.426851 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f6fb5e208cd371ce176a8ec4882a197fd8a7669e041b2c6a5eae598f454508d\": container with ID starting with 6f6fb5e208cd371ce176a8ec4882a197fd8a7669e041b2c6a5eae598f454508d not found: ID does not exist" containerID="6f6fb5e208cd371ce176a8ec4882a197fd8a7669e041b2c6a5eae598f454508d" Jul 07 00:15:39.426922 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.426883 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f6fb5e208cd371ce176a8ec4882a197fd8a7669e041b2c6a5eae598f454508d"} err="failed to get container status \"6f6fb5e208cd371ce176a8ec4882a197fd8a7669e041b2c6a5eae598f454508d\": rpc error: code = NotFound desc = could not find container \"6f6fb5e208cd371ce176a8ec4882a197fd8a7669e041b2c6a5eae598f454508d\": container with ID starting with 6f6fb5e208cd371ce176a8ec4882a197fd8a7669e041b2c6a5eae598f454508d not found: ID does not exist" Jul 07 00:15:39.439716 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.439690 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg"] Jul 07 00:15:39.444549 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:39.444523 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-2a6aa737-openshift-default-78fbb5dbcf-4ldqg"] Jul 07 00:15:40.304026 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:15:40.303992 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed" path="/var/lib/kubelet/pods/9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed/volumes" Jul 07 00:17:15.701078 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.700989 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn"] Jul 07 00:17:15.701567 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.701486 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed" containerName="istio-proxy" Jul 07 00:17:15.701567 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.701502 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed" containerName="istio-proxy" Jul 07 00:17:15.701641 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.701582 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c3c0721-f35f-4dbb-8eb0-fbc9253080b9" containerName="istio-proxy" Jul 07 00:17:15.701641 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.701589 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="9738ebe4-6ca2-48bf-8cfb-41d12bfd18ed" containerName="istio-proxy" Jul 07 00:17:15.704792 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.704774 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:15.707776 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.707753 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-4a8aca1c-openshift-default-dockercfg-n7vx4\"" Jul 07 00:17:15.726095 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.726063 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn"] Jul 07 00:17:15.854586 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.854545 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7f380e3-3133-4d7f-be32-a782cc1b709e-istio-podinfo\") pod \"e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:15.854771 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.854594 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzgr8\" (UniqueName: \"kubernetes.io/projected/a7f380e3-3133-4d7f-be32-a782cc1b709e-kube-api-access-tzgr8\") pod \"e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:15.854771 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.854621 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-istio-envoy\") pod \"e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:15.854771 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.854672 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7f380e3-3133-4d7f-be32-a782cc1b709e-istiod-ca-cert\") pod \"e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:15.854771 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.854716 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-credential-socket\") pod \"e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:15.854771 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.854733 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-workload-certs\") pod \"e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:15.854771 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.854756 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-istio-data\") pod \"e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:15.854977 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.854821 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-workload-socket\") pod \"e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:15.854977 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.854846 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7f380e3-3133-4d7f-be32-a782cc1b709e-istio-token\") pod \"e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:15.955916 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.955822 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7f380e3-3133-4d7f-be32-a782cc1b709e-istio-podinfo\") pod \"e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:15.955916 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.955870 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzgr8\" (UniqueName: \"kubernetes.io/projected/a7f380e3-3133-4d7f-be32-a782cc1b709e-kube-api-access-tzgr8\") pod \"e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:15.955916 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.955891 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-istio-envoy\") pod \"e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:15.955916 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.955915 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7f380e3-3133-4d7f-be32-a782cc1b709e-istiod-ca-cert\") pod \"e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:15.956276 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.955947 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-credential-socket\") pod \"e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:15.956276 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.956069 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-workload-certs\") pod \"e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:15.956276 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.956117 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-istio-data\") pod \"e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:15.956276 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.956188 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-workload-socket\") pod \"e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:15.956276 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.956217 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7f380e3-3133-4d7f-be32-a782cc1b709e-istio-token\") pod \"e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:15.956590 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.956436 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-credential-socket\") pod \"e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:15.956590 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.956540 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-istio-data\") pod \"e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:15.956590 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.956573 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-workload-certs\") pod \"e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:15.956721 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.956629 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-workload-socket\") pod \"e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:15.957543 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.957417 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7f380e3-3133-4d7f-be32-a782cc1b709e-istiod-ca-cert\") pod \"e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:15.958798 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.958774 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-istio-envoy\") pod \"e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:15.959393 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.959368 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7f380e3-3133-4d7f-be32-a782cc1b709e-istio-podinfo\") pod \"e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:15.964800 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.964770 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7f380e3-3133-4d7f-be32-a782cc1b709e-istio-token\") pod \"e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:15.964925 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:15.964910 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzgr8\" (UniqueName: \"kubernetes.io/projected/a7f380e3-3133-4d7f-be32-a782cc1b709e-kube-api-access-tzgr8\") pod \"e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:16.015495 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:16.015453 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:16.173269 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:16.173239 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn"] Jul 07 00:17:16.174608 ip-10-0-136-25 kubenswrapper[2553]: W0707 00:17:16.174582 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7f380e3_3133_4d7f_be32_a782cc1b709e.slice/crio-c1117652810bbf7b94aa7ad3f43c7ff4e2a8ba4f219efa9738b5f491095c7ac7 WatchSource:0}: Error finding container c1117652810bbf7b94aa7ad3f43c7ff4e2a8ba4f219efa9738b5f491095c7ac7: Status 404 returned error can't find the container with id c1117652810bbf7b94aa7ad3f43c7ff4e2a8ba4f219efa9738b5f491095c7ac7 Jul 07 00:17:16.176895 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:16.176855 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:17:16.176971 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:16.176949 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:17:16.177037 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:16.177019 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:17:16.978159 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:16.978121 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" event={"ID":"a7f380e3-3133-4d7f-be32-a782cc1b709e","Type":"ContainerStarted","Data":"4a87158cc1831f73698dbafc82dd8440b91023a0c4f8143e6805673e93171c89"} Jul 07 00:17:16.978587 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:16.978186 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" event={"ID":"a7f380e3-3133-4d7f-be32-a782cc1b709e","Type":"ContainerStarted","Data":"c1117652810bbf7b94aa7ad3f43c7ff4e2a8ba4f219efa9738b5f491095c7ac7"} Jul 07 00:17:16.998630 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:16.998575 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" podStartSLOduration=1.998556912 podStartE2EDuration="1.998556912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:17:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:17:16.997806476 +0000 UTC m=+1719.207536024" watchObservedRunningTime="2026-07-07 00:17:16.998556912 +0000 UTC m=+1719.208286436" Jul 07 00:17:17.016212 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:17.016179 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:17.022728 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:17.022696 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:17.985630 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:17.985595 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:17.986593 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:17.986575 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:31.087730 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:31.087696 2553 generic.go:356] "Generic (PLEG): container finished" podID="a7f380e3-3133-4d7f-be32-a782cc1b709e" containerID="4a87158cc1831f73698dbafc82dd8440b91023a0c4f8143e6805673e93171c89" exitCode=0 Jul 07 00:17:31.088189 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:31.087768 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" event={"ID":"a7f380e3-3133-4d7f-be32-a782cc1b709e","Type":"ContainerDied","Data":"4a87158cc1831f73698dbafc82dd8440b91023a0c4f8143e6805673e93171c89"} Jul 07 00:17:31.088189 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:31.088164 2553 scope.go:117] "RemoveContainer" containerID="4a87158cc1831f73698dbafc82dd8440b91023a0c4f8143e6805673e93171c89" Jul 07 00:17:31.089199 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:31.089175 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:17:31.089273 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:31.089241 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:17:31.089349 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:31.089277 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:17:32.095848 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:32.095815 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" event={"ID":"a7f380e3-3133-4d7f-be32-a782cc1b709e","Type":"ContainerStarted","Data":"6fe2fc3ce50c7b215f1ed07aafbf5bbe354686a78d2dd8aa49af4d459af56c81"} Jul 07 00:17:32.986057 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:32.986016 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:33.016659 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:33.016619 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:33.018074 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:33.018035 2553 patch_prober.go:28] interesting pod/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Jul 07 00:17:33.018212 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:33.018102 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" podUID="a7f380e3-3133-4d7f-be32-a782cc1b709e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Jul 07 00:17:34.016578 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:34.016534 2553 patch_prober.go:28] interesting pod/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Jul 07 00:17:34.017081 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:34.016607 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" podUID="a7f380e3-3133-4d7f-be32-a782cc1b709e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Jul 07 00:17:35.016794 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:35.016751 2553 patch_prober.go:28] interesting pod/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Jul 07 00:17:35.017328 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:35.016827 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" podUID="a7f380e3-3133-4d7f-be32-a782cc1b709e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Jul 07 00:17:36.016390 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:36.016343 2553 patch_prober.go:28] interesting pod/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Jul 07 00:17:36.016634 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:36.016407 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" podUID="a7f380e3-3133-4d7f-be32-a782cc1b709e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Jul 07 00:17:37.021070 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:37.021037 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:37.023596 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:37.023562 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:37.206795 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:37.206760 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn"] Jul 07 00:17:38.131053 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:38.131001 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" podUID="a7f380e3-3133-4d7f-be32-a782cc1b709e" containerName="istio-proxy" containerID="cri-o://6fe2fc3ce50c7b215f1ed07aafbf5bbe354686a78d2dd8aa49af4d459af56c81" gracePeriod=30 Jul 07 00:17:43.389660 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:43.389590 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:43.531673 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:43.531646 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-workload-socket\") pod \"a7f380e3-3133-4d7f-be32-a782cc1b709e\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " Jul 07 00:17:43.531673 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:43.531678 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-credential-socket\") pod \"a7f380e3-3133-4d7f-be32-a782cc1b709e\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " Jul 07 00:17:43.531891 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:43.531723 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzgr8\" (UniqueName: \"kubernetes.io/projected/a7f380e3-3133-4d7f-be32-a782cc1b709e-kube-api-access-tzgr8\") pod \"a7f380e3-3133-4d7f-be32-a782cc1b709e\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " Jul 07 00:17:43.531891 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:43.531758 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7f380e3-3133-4d7f-be32-a782cc1b709e-istio-podinfo\") pod \"a7f380e3-3133-4d7f-be32-a782cc1b709e\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " Jul 07 00:17:43.531891 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:43.531784 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7f380e3-3133-4d7f-be32-a782cc1b709e-istiod-ca-cert\") pod \"a7f380e3-3133-4d7f-be32-a782cc1b709e\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " Jul 07 00:17:43.531891 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:43.531857 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-workload-certs\") pod \"a7f380e3-3133-4d7f-be32-a782cc1b709e\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " Jul 07 00:17:43.531891 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:43.531885 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7f380e3-3133-4d7f-be32-a782cc1b709e-istio-token\") pod \"a7f380e3-3133-4d7f-be32-a782cc1b709e\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " Jul 07 00:17:43.532160 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:43.531912 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-istio-data\") pod \"a7f380e3-3133-4d7f-be32-a782cc1b709e\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " Jul 07 00:17:43.532160 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:43.531993 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-istio-envoy\") pod \"a7f380e3-3133-4d7f-be32-a782cc1b709e\" (UID: \"a7f380e3-3133-4d7f-be32-a782cc1b709e\") " Jul 07 00:17:43.532160 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:43.531907 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a7f380e3-3133-4d7f-be32-a782cc1b709e" (UID: "a7f380e3-3133-4d7f-be32-a782cc1b709e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:17:43.532160 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:43.532044 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a7f380e3-3133-4d7f-be32-a782cc1b709e" (UID: "a7f380e3-3133-4d7f-be32-a782cc1b709e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:17:43.532405 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:43.532326 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7f380e3-3133-4d7f-be32-a782cc1b709e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a7f380e3-3133-4d7f-be32-a782cc1b709e" (UID: "a7f380e3-3133-4d7f-be32-a782cc1b709e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:17:43.532405 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:43.532345 2553 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-workload-socket\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:17:43.532405 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:43.532364 2553 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-credential-socket\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:17:43.532571 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:43.532508 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a7f380e3-3133-4d7f-be32-a782cc1b709e" (UID: "a7f380e3-3133-4d7f-be32-a782cc1b709e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:17:43.532896 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:43.532873 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-istio-data" (OuterVolumeSpecName: "istio-data") pod "a7f380e3-3133-4d7f-be32-a782cc1b709e" (UID: "a7f380e3-3133-4d7f-be32-a782cc1b709e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:17:43.534104 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:43.534064 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a7f380e3-3133-4d7f-be32-a782cc1b709e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a7f380e3-3133-4d7f-be32-a782cc1b709e" (UID: "a7f380e3-3133-4d7f-be32-a782cc1b709e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 00:17:43.534104 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:43.534064 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7f380e3-3133-4d7f-be32-a782cc1b709e-kube-api-access-tzgr8" (OuterVolumeSpecName: "kube-api-access-tzgr8") pod "a7f380e3-3133-4d7f-be32-a782cc1b709e" (UID: "a7f380e3-3133-4d7f-be32-a782cc1b709e"). InnerVolumeSpecName "kube-api-access-tzgr8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:17:43.534427 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:43.534400 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a7f380e3-3133-4d7f-be32-a782cc1b709e" (UID: "a7f380e3-3133-4d7f-be32-a782cc1b709e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:17:43.534635 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:43.534615 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7f380e3-3133-4d7f-be32-a782cc1b709e-istio-token" (OuterVolumeSpecName: "istio-token") pod "a7f380e3-3133-4d7f-be32-a782cc1b709e" (UID: "a7f380e3-3133-4d7f-be32-a782cc1b709e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:17:43.633406 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:43.633371 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzgr8\" (UniqueName: \"kubernetes.io/projected/a7f380e3-3133-4d7f-be32-a782cc1b709e-kube-api-access-tzgr8\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:17:43.633406 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:43.633403 2553 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7f380e3-3133-4d7f-be32-a782cc1b709e-istio-podinfo\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:17:43.633406 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:43.633415 2553 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7f380e3-3133-4d7f-be32-a782cc1b709e-istiod-ca-cert\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:17:43.633652 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:43.633427 2553 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-workload-certs\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:17:43.633652 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:43.633438 2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7f380e3-3133-4d7f-be32-a782cc1b709e-istio-token\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:17:43.633652 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:43.633449 2553 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-istio-data\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:17:43.633652 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:43.633458 2553 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7f380e3-3133-4d7f-be32-a782cc1b709e-istio-envoy\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:17:44.167436 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:44.167398 2553 generic.go:356] "Generic (PLEG): container finished" podID="a7f380e3-3133-4d7f-be32-a782cc1b709e" containerID="6fe2fc3ce50c7b215f1ed07aafbf5bbe354686a78d2dd8aa49af4d459af56c81" exitCode=0 Jul 07 00:17:44.167629 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:44.167446 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" Jul 07 00:17:44.167629 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:44.167480 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" event={"ID":"a7f380e3-3133-4d7f-be32-a782cc1b709e","Type":"ContainerDied","Data":"6fe2fc3ce50c7b215f1ed07aafbf5bbe354686a78d2dd8aa49af4d459af56c81"} Jul 07 00:17:44.167629 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:44.167541 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn" event={"ID":"a7f380e3-3133-4d7f-be32-a782cc1b709e","Type":"ContainerDied","Data":"c1117652810bbf7b94aa7ad3f43c7ff4e2a8ba4f219efa9738b5f491095c7ac7"} Jul 07 00:17:44.167629 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:44.167579 2553 scope.go:117] "RemoveContainer" containerID="6fe2fc3ce50c7b215f1ed07aafbf5bbe354686a78d2dd8aa49af4d459af56c81" Jul 07 00:17:44.184405 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:44.184383 2553 scope.go:117] "RemoveContainer" containerID="4a87158cc1831f73698dbafc82dd8440b91023a0c4f8143e6805673e93171c89" Jul 07 00:17:44.196384 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:44.196359 2553 scope.go:117] "RemoveContainer" containerID="6fe2fc3ce50c7b215f1ed07aafbf5bbe354686a78d2dd8aa49af4d459af56c81" Jul 07 00:17:44.196704 ip-10-0-136-25 kubenswrapper[2553]: E0707 00:17:44.196684 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fe2fc3ce50c7b215f1ed07aafbf5bbe354686a78d2dd8aa49af4d459af56c81\": container with ID starting with 6fe2fc3ce50c7b215f1ed07aafbf5bbe354686a78d2dd8aa49af4d459af56c81 not found: ID does not exist" containerID="6fe2fc3ce50c7b215f1ed07aafbf5bbe354686a78d2dd8aa49af4d459af56c81" Jul 07 00:17:44.196783 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:44.196713 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fe2fc3ce50c7b215f1ed07aafbf5bbe354686a78d2dd8aa49af4d459af56c81"} err="failed to get container status \"6fe2fc3ce50c7b215f1ed07aafbf5bbe354686a78d2dd8aa49af4d459af56c81\": rpc error: code = NotFound desc = could not find container \"6fe2fc3ce50c7b215f1ed07aafbf5bbe354686a78d2dd8aa49af4d459af56c81\": container with ID starting with 6fe2fc3ce50c7b215f1ed07aafbf5bbe354686a78d2dd8aa49af4d459af56c81 not found: ID does not exist" Jul 07 00:17:44.196783 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:44.196733 2553 scope.go:117] "RemoveContainer" containerID="4a87158cc1831f73698dbafc82dd8440b91023a0c4f8143e6805673e93171c89" Jul 07 00:17:44.197004 ip-10-0-136-25 kubenswrapper[2553]: E0707 00:17:44.196985 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a87158cc1831f73698dbafc82dd8440b91023a0c4f8143e6805673e93171c89\": container with ID starting with 4a87158cc1831f73698dbafc82dd8440b91023a0c4f8143e6805673e93171c89 not found: ID does not exist" containerID="4a87158cc1831f73698dbafc82dd8440b91023a0c4f8143e6805673e93171c89" Jul 07 00:17:44.197068 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:44.197008 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a87158cc1831f73698dbafc82dd8440b91023a0c4f8143e6805673e93171c89"} err="failed to get container status \"4a87158cc1831f73698dbafc82dd8440b91023a0c4f8143e6805673e93171c89\": rpc error: code = NotFound desc = could not find container \"4a87158cc1831f73698dbafc82dd8440b91023a0c4f8143e6805673e93171c89\": container with ID starting with 4a87158cc1831f73698dbafc82dd8440b91023a0c4f8143e6805673e93171c89 not found: ID does not exist" Jul 07 00:17:44.198997 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:44.198972 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn"] Jul 07 00:17:44.203632 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:44.203607 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-4a8aca1c-openshift-default-5c6dc58dcf-6zpfn"] Jul 07 00:17:44.303669 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:44.303632 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7f380e3-3133-4d7f-be32-a782cc1b709e" path="/var/lib/kubelet/pods/a7f380e3-3133-4d7f-be32-a782cc1b709e/volumes" Jul 07 00:17:56.891616 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:56.891580 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7"] Jul 07 00:17:56.892278 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:56.892254 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7f380e3-3133-4d7f-be32-a782cc1b709e" containerName="istio-proxy" Jul 07 00:17:56.892404 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:56.892280 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7f380e3-3133-4d7f-be32-a782cc1b709e" containerName="istio-proxy" Jul 07 00:17:56.892463 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:56.892446 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7f380e3-3133-4d7f-be32-a782cc1b709e" containerName="istio-proxy" Jul 07 00:17:56.892514 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:56.892465 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7f380e3-3133-4d7f-be32-a782cc1b709e" containerName="istio-proxy" Jul 07 00:17:56.896972 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:56.896951 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:56.899545 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:56.899521 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-1ddddfee-openshift-default-dockercfg-ks5kj\"" Jul 07 00:17:56.907048 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:56.907021 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7"] Jul 07 00:17:57.068577 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.068540 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a11b345-a320-454f-83c8-a634bb7d78b4-istio-token\") pod \"e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:57.068762 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.068590 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a11b345-a320-454f-83c8-a634bb7d78b4-istio-podinfo\") pod \"e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:57.068762 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.068620 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a11b345-a320-454f-83c8-a634bb7d78b4-istiod-ca-cert\") pod \"e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:57.068762 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.068647 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjfk8\" (UniqueName: \"kubernetes.io/projected/6a11b345-a320-454f-83c8-a634bb7d78b4-kube-api-access-mjfk8\") pod \"e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:57.068762 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.068683 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-istio-data\") pod \"e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:57.068762 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.068720 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-workload-socket\") pod \"e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:57.068762 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.068736 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-workload-certs\") pod \"e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:57.068958 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.068766 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-istio-envoy\") pod \"e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:57.068958 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.068855 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-credential-socket\") pod \"e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:57.170356 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.170286 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a11b345-a320-454f-83c8-a634bb7d78b4-istio-podinfo\") pod \"e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:57.170356 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.170363 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a11b345-a320-454f-83c8-a634bb7d78b4-istiod-ca-cert\") pod \"e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:57.170660 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.170382 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjfk8\" (UniqueName: \"kubernetes.io/projected/6a11b345-a320-454f-83c8-a634bb7d78b4-kube-api-access-mjfk8\") pod \"e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:57.170660 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.170415 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-istio-data\") pod \"e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:57.170660 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.170447 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-workload-socket\") pod \"e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:57.170660 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.170462 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-workload-certs\") pod \"e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:57.170660 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.170505 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-istio-envoy\") pod \"e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:57.170660 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.170633 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-credential-socket\") pod \"e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:57.171060 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.170735 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a11b345-a320-454f-83c8-a634bb7d78b4-istio-token\") pod \"e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:57.171060 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.170943 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-workload-socket\") pod \"e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:57.171060 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.171028 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-workload-certs\") pod \"e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:57.171190 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.171171 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-credential-socket\") pod \"e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:57.171234 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.171220 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-istio-data\") pod \"e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:57.171472 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.171454 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a11b345-a320-454f-83c8-a634bb7d78b4-istiod-ca-cert\") pod \"e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:57.173234 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.173218 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-istio-envoy\") pod \"e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:57.173585 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.173563 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a11b345-a320-454f-83c8-a634bb7d78b4-istio-podinfo\") pod \"e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:57.182815 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.182786 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a11b345-a320-454f-83c8-a634bb7d78b4-istio-token\") pod \"e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:57.183013 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.182993 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjfk8\" (UniqueName: \"kubernetes.io/projected/6a11b345-a320-454f-83c8-a634bb7d78b4-kube-api-access-mjfk8\") pod \"e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:57.208563 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.208518 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:57.372784 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.372754 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7"] Jul 07 00:17:57.375383 ip-10-0-136-25 kubenswrapper[2553]: W0707 00:17:57.375355 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a11b345_a320_454f_83c8_a634bb7d78b4.slice/crio-391a0536a13adb362fea2edcb9bd741cf920953a83db6a2265b5e072ef84b6ef WatchSource:0}: Error finding container 391a0536a13adb362fea2edcb9bd741cf920953a83db6a2265b5e072ef84b6ef: Status 404 returned error can't find the container with id 391a0536a13adb362fea2edcb9bd741cf920953a83db6a2265b5e072ef84b6ef Jul 07 00:17:57.377535 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.377501 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:17:57.377642 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.377568 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:17:57.377642 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:57.377600 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:17:58.258156 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:58.258112 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" event={"ID":"6a11b345-a320-454f-83c8-a634bb7d78b4","Type":"ContainerStarted","Data":"5232ddcb616eaa13825c1f8bb3ca2dfe6a002f29841b65e9086e6d4f72b0f244"} Jul 07 00:17:58.258591 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:58.258182 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" event={"ID":"6a11b345-a320-454f-83c8-a634bb7d78b4","Type":"ContainerStarted","Data":"391a0536a13adb362fea2edcb9bd741cf920953a83db6a2265b5e072ef84b6ef"} Jul 07 00:17:58.279497 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:58.279445 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" podStartSLOduration=2.279426822 podStartE2EDuration="2.279426822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:17:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:17:58.277376809 +0000 UTC m=+1760.487106351" watchObservedRunningTime="2026-07-07 00:17:58.279426822 +0000 UTC m=+1760.489156342" Jul 07 00:17:59.209442 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:59.209401 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:59.214776 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:59.214751 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:59.264632 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:59.264594 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:17:59.265480 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:17:59.265458 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:18:14.917010 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:18:14.915663 2553 patch_prober.go:28] interesting pod/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": read tcp 10.132.0.2:48036->10.132.0.70:15021: read: connection reset by peer" start-of-body= Jul 07 00:18:14.917010 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:18:14.915749 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" podUID="6a11b345-a320-454f-83c8-a634bb7d78b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": read tcp 10.132.0.2:48036->10.132.0.70:15021: read: connection reset by peer" Jul 07 00:18:15.385019 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:18:15.384928 2553 generic.go:356] "Generic (PLEG): container finished" podID="6a11b345-a320-454f-83c8-a634bb7d78b4" containerID="5232ddcb616eaa13825c1f8bb3ca2dfe6a002f29841b65e9086e6d4f72b0f244" exitCode=0 Jul 07 00:18:15.385019 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:18:15.385000 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" event={"ID":"6a11b345-a320-454f-83c8-a634bb7d78b4","Type":"ContainerDied","Data":"5232ddcb616eaa13825c1f8bb3ca2dfe6a002f29841b65e9086e6d4f72b0f244"} Jul 07 00:18:15.385478 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:18:15.385464 2553 scope.go:117] "RemoveContainer" containerID="5232ddcb616eaa13825c1f8bb3ca2dfe6a002f29841b65e9086e6d4f72b0f244" Jul 07 00:18:15.386545 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:18:15.386514 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:18:15.386650 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:18:15.386585 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:18:15.386650 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:18:15.386615 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:18:16.392949 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:18:16.392914 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" event={"ID":"6a11b345-a320-454f-83c8-a634bb7d78b4","Type":"ContainerStarted","Data":"6b7e17df3a87c23dd7dc9d865ed48e859247f5c6e6fda2a590f1a050242665fd"} Jul 07 00:18:17.209464 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:18:17.209430 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:18:17.209726 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:18:17.209591 2553 patch_prober.go:28] interesting pod/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Jul 07 00:18:17.209726 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:18:17.209636 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" podUID="6a11b345-a320-454f-83c8-a634bb7d78b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Jul 07 00:18:18.209743 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:18:18.209707 2553 patch_prober.go:28] interesting pod/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Jul 07 00:18:18.210131 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:18:18.209761 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" podUID="6a11b345-a320-454f-83c8-a634bb7d78b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Jul 07 00:18:19.209663 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:18:19.209624 2553 patch_prober.go:28] interesting pod/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Jul 07 00:18:19.209851 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:18:19.209685 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" podUID="6a11b345-a320-454f-83c8-a634bb7d78b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Jul 07 00:18:20.209523 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:18:20.209486 2553 patch_prober.go:28] interesting pod/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Jul 07 00:18:20.209698 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:18:20.209544 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" podUID="6a11b345-a320-454f-83c8-a634bb7d78b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Jul 07 00:18:21.212827 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:18:21.212796 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:18:21.213284 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:18:21.213264 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:18:21.214004 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:18:21.213988 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:18:38.369890 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:18:38.369858 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4z8vq_3253bdfe-5248-4d4c-9864-11660eb042ca/ovn-acl-logging/0.log" Jul 07 00:18:38.370648 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:18:38.370628 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4z8vq_3253bdfe-5248-4d4c-9864-11660eb042ca/ovn-acl-logging/0.log" Jul 07 00:20:10.007965 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:10.007881 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7"] Jul 07 00:20:10.008518 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:10.008110 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" podUID="6a11b345-a320-454f-83c8-a634bb7d78b4" containerName="istio-proxy" containerID="cri-o://6b7e17df3a87c23dd7dc9d865ed48e859247f5c6e6fda2a590f1a050242665fd" gracePeriod=30 Jul 07 00:20:15.272503 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:15.272431 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:20:15.337965 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:15.337930 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-istio-envoy\") pod \"6a11b345-a320-454f-83c8-a634bb7d78b4\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " Jul 07 00:20:15.338158 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:15.338005 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-istio-data\") pod \"6a11b345-a320-454f-83c8-a634bb7d78b4\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " Jul 07 00:20:15.338158 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:15.338030 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-workload-certs\") pod \"6a11b345-a320-454f-83c8-a634bb7d78b4\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " Jul 07 00:20:15.338158 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:15.338087 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a11b345-a320-454f-83c8-a634bb7d78b4-istio-token\") pod \"6a11b345-a320-454f-83c8-a634bb7d78b4\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " Jul 07 00:20:15.338158 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:15.338124 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjfk8\" (UniqueName: \"kubernetes.io/projected/6a11b345-a320-454f-83c8-a634bb7d78b4-kube-api-access-mjfk8\") pod \"6a11b345-a320-454f-83c8-a634bb7d78b4\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " Jul 07 00:20:15.338414 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:15.338193 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a11b345-a320-454f-83c8-a634bb7d78b4-istiod-ca-cert\") pod \"6a11b345-a320-454f-83c8-a634bb7d78b4\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " Jul 07 00:20:15.338414 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:15.338235 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-credential-socket\") pod \"6a11b345-a320-454f-83c8-a634bb7d78b4\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " Jul 07 00:20:15.338414 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:15.338307 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a11b345-a320-454f-83c8-a634bb7d78b4-istio-podinfo\") pod \"6a11b345-a320-454f-83c8-a634bb7d78b4\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " Jul 07 00:20:15.338414 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:15.338365 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-workload-socket\") pod \"6a11b345-a320-454f-83c8-a634bb7d78b4\" (UID: \"6a11b345-a320-454f-83c8-a634bb7d78b4\") " Jul 07 00:20:15.338615 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:15.338443 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6a11b345-a320-454f-83c8-a634bb7d78b4" (UID: "6a11b345-a320-454f-83c8-a634bb7d78b4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:20:15.338916 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:15.338597 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6a11b345-a320-454f-83c8-a634bb7d78b4" (UID: "6a11b345-a320-454f-83c8-a634bb7d78b4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:20:15.339082 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:15.339037 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a11b345-a320-454f-83c8-a634bb7d78b4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6a11b345-a320-454f-83c8-a634bb7d78b4" (UID: "6a11b345-a320-454f-83c8-a634bb7d78b4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:20:15.339205 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:15.339110 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-istio-data" (OuterVolumeSpecName: "istio-data") pod "6a11b345-a320-454f-83c8-a634bb7d78b4" (UID: "6a11b345-a320-454f-83c8-a634bb7d78b4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:20:15.339205 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:15.339175 2553 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-workload-certs\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:20:15.339514 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:15.339210 2553 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-credential-socket\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:20:15.339514 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:15.339364 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6a11b345-a320-454f-83c8-a634bb7d78b4" (UID: "6a11b345-a320-454f-83c8-a634bb7d78b4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:20:15.340647 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:15.340624 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6a11b345-a320-454f-83c8-a634bb7d78b4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6a11b345-a320-454f-83c8-a634bb7d78b4" (UID: "6a11b345-a320-454f-83c8-a634bb7d78b4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 00:20:15.340815 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:15.340791 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a11b345-a320-454f-83c8-a634bb7d78b4-istio-token" (OuterVolumeSpecName: "istio-token") pod "6a11b345-a320-454f-83c8-a634bb7d78b4" (UID: "6a11b345-a320-454f-83c8-a634bb7d78b4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:20:15.340893 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:15.340812 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6a11b345-a320-454f-83c8-a634bb7d78b4" (UID: "6a11b345-a320-454f-83c8-a634bb7d78b4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:20:15.340893 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:15.340838 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a11b345-a320-454f-83c8-a634bb7d78b4-kube-api-access-mjfk8" (OuterVolumeSpecName: "kube-api-access-mjfk8") pod "6a11b345-a320-454f-83c8-a634bb7d78b4" (UID: "6a11b345-a320-454f-83c8-a634bb7d78b4"). InnerVolumeSpecName "kube-api-access-mjfk8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:20:15.440056 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:15.440018 2553 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-istio-data\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:20:15.440056 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:15.440044 2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a11b345-a320-454f-83c8-a634bb7d78b4-istio-token\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:20:15.440056 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:15.440054 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjfk8\" (UniqueName: \"kubernetes.io/projected/6a11b345-a320-454f-83c8-a634bb7d78b4-kube-api-access-mjfk8\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:20:15.440056 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:15.440062 2553 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a11b345-a320-454f-83c8-a634bb7d78b4-istiod-ca-cert\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:20:15.440394 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:15.440070 2553 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a11b345-a320-454f-83c8-a634bb7d78b4-istio-podinfo\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:20:15.440394 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:15.440080 2553 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-workload-socket\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:20:15.440394 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:15.440096 2553 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a11b345-a320-454f-83c8-a634bb7d78b4-istio-envoy\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:20:16.116164 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:16.116127 2553 generic.go:356] "Generic (PLEG): container finished" podID="6a11b345-a320-454f-83c8-a634bb7d78b4" containerID="6b7e17df3a87c23dd7dc9d865ed48e859247f5c6e6fda2a590f1a050242665fd" exitCode=0 Jul 07 00:20:16.116468 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:16.116200 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" Jul 07 00:20:16.116468 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:16.116212 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" event={"ID":"6a11b345-a320-454f-83c8-a634bb7d78b4","Type":"ContainerDied","Data":"6b7e17df3a87c23dd7dc9d865ed48e859247f5c6e6fda2a590f1a050242665fd"} Jul 07 00:20:16.116468 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:16.116269 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7" event={"ID":"6a11b345-a320-454f-83c8-a634bb7d78b4","Type":"ContainerDied","Data":"391a0536a13adb362fea2edcb9bd741cf920953a83db6a2265b5e072ef84b6ef"} Jul 07 00:20:16.116468 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:16.116313 2553 scope.go:117] "RemoveContainer" containerID="6b7e17df3a87c23dd7dc9d865ed48e859247f5c6e6fda2a590f1a050242665fd" Jul 07 00:20:16.127242 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:16.127118 2553 scope.go:117] "RemoveContainer" containerID="5232ddcb616eaa13825c1f8bb3ca2dfe6a002f29841b65e9086e6d4f72b0f244" Jul 07 00:20:16.136870 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:16.136819 2553 scope.go:117] "RemoveContainer" containerID="6b7e17df3a87c23dd7dc9d865ed48e859247f5c6e6fda2a590f1a050242665fd" Jul 07 00:20:16.137166 ip-10-0-136-25 kubenswrapper[2553]: E0707 00:20:16.137139 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b7e17df3a87c23dd7dc9d865ed48e859247f5c6e6fda2a590f1a050242665fd\": container with ID starting with 6b7e17df3a87c23dd7dc9d865ed48e859247f5c6e6fda2a590f1a050242665fd not found: ID does not exist" containerID="6b7e17df3a87c23dd7dc9d865ed48e859247f5c6e6fda2a590f1a050242665fd" Jul 07 00:20:16.137239 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:16.137167 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b7e17df3a87c23dd7dc9d865ed48e859247f5c6e6fda2a590f1a050242665fd"} err="failed to get container status \"6b7e17df3a87c23dd7dc9d865ed48e859247f5c6e6fda2a590f1a050242665fd\": rpc error: code = NotFound desc = could not find container \"6b7e17df3a87c23dd7dc9d865ed48e859247f5c6e6fda2a590f1a050242665fd\": container with ID starting with 6b7e17df3a87c23dd7dc9d865ed48e859247f5c6e6fda2a590f1a050242665fd not found: ID does not exist" Jul 07 00:20:16.137239 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:16.137186 2553 scope.go:117] "RemoveContainer" containerID="5232ddcb616eaa13825c1f8bb3ca2dfe6a002f29841b65e9086e6d4f72b0f244" Jul 07 00:20:16.137600 ip-10-0-136-25 kubenswrapper[2553]: E0707 00:20:16.137579 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5232ddcb616eaa13825c1f8bb3ca2dfe6a002f29841b65e9086e6d4f72b0f244\": container with ID starting with 5232ddcb616eaa13825c1f8bb3ca2dfe6a002f29841b65e9086e6d4f72b0f244 not found: ID does not exist" containerID="5232ddcb616eaa13825c1f8bb3ca2dfe6a002f29841b65e9086e6d4f72b0f244" Jul 07 00:20:16.137683 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:16.137606 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5232ddcb616eaa13825c1f8bb3ca2dfe6a002f29841b65e9086e6d4f72b0f244"} err="failed to get container status \"5232ddcb616eaa13825c1f8bb3ca2dfe6a002f29841b65e9086e6d4f72b0f244\": rpc error: code = NotFound desc = could not find container \"5232ddcb616eaa13825c1f8bb3ca2dfe6a002f29841b65e9086e6d4f72b0f244\": container with ID starting with 5232ddcb616eaa13825c1f8bb3ca2dfe6a002f29841b65e9086e6d4f72b0f244 not found: ID does not exist" Jul 07 00:20:16.144238 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:16.144217 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7"] Jul 07 00:20:16.146895 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:16.146876 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-1ddddfee-openshift-default-86cc5cd56c-qqhn7"] Jul 07 00:20:16.303413 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:20:16.303380 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a11b345-a320-454f-83c8-a634bb7d78b4" path="/var/lib/kubelet/pods/6a11b345-a320-454f-83c8-a634bb7d78b4/volumes" Jul 07 00:21:05.518750 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.518712 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f"] Jul 07 00:21:05.519396 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.519364 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a11b345-a320-454f-83c8-a634bb7d78b4" containerName="istio-proxy" Jul 07 00:21:05.519396 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.519385 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a11b345-a320-454f-83c8-a634bb7d78b4" containerName="istio-proxy" Jul 07 00:21:05.519531 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.519409 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7f380e3-3133-4d7f-be32-a782cc1b709e" containerName="istio-proxy" Jul 07 00:21:05.519531 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.519417 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7f380e3-3133-4d7f-be32-a782cc1b709e" containerName="istio-proxy" Jul 07 00:21:05.519531 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.519439 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a11b345-a320-454f-83c8-a634bb7d78b4" containerName="istio-proxy" Jul 07 00:21:05.519531 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.519449 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a11b345-a320-454f-83c8-a634bb7d78b4" containerName="istio-proxy" Jul 07 00:21:05.519721 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.519569 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a11b345-a320-454f-83c8-a634bb7d78b4" containerName="istio-proxy" Jul 07 00:21:05.523044 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.523020 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:05.526070 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.526044 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-b48493-openshift-default-dockercfg-8j827\"" Jul 07 00:21:05.541860 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.541830 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f"] Jul 07 00:21:05.613411 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.613365 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-workload-certs\") pod \"e2e-inf-a-b48493-openshift-default-64848454f-jp65f\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:05.613741 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.613719 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istio-podinfo\") pod \"e2e-inf-a-b48493-openshift-default-64848454f-jp65f\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:05.613928 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.613911 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmlvt\" (UniqueName: \"kubernetes.io/projected/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-kube-api-access-dmlvt\") pod \"e2e-inf-a-b48493-openshift-default-64848454f-jp65f\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:05.614083 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.614067 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istio-token\") pod \"e2e-inf-a-b48493-openshift-default-64848454f-jp65f\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:05.614365 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.614343 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-workload-socket\") pod \"e2e-inf-a-b48493-openshift-default-64848454f-jp65f\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:05.614742 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.614375 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istio-data\") pod \"e2e-inf-a-b48493-openshift-default-64848454f-jp65f\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:05.614742 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.614402 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-credential-socket\") pod \"e2e-inf-a-b48493-openshift-default-64848454f-jp65f\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:05.614742 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.614450 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istio-envoy\") pod \"e2e-inf-a-b48493-openshift-default-64848454f-jp65f\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:05.614742 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.614484 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istiod-ca-cert\") pod \"e2e-inf-a-b48493-openshift-default-64848454f-jp65f\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:05.715861 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.715813 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-workload-certs\") pod \"e2e-inf-a-b48493-openshift-default-64848454f-jp65f\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:05.716091 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.715873 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istio-podinfo\") pod \"e2e-inf-a-b48493-openshift-default-64848454f-jp65f\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:05.716091 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.715907 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmlvt\" (UniqueName: \"kubernetes.io/projected/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-kube-api-access-dmlvt\") pod \"e2e-inf-a-b48493-openshift-default-64848454f-jp65f\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:05.716091 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.715943 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istio-token\") pod \"e2e-inf-a-b48493-openshift-default-64848454f-jp65f\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:05.716091 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.716011 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-workload-socket\") pod \"e2e-inf-a-b48493-openshift-default-64848454f-jp65f\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:05.716091 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.716034 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istio-data\") pod \"e2e-inf-a-b48493-openshift-default-64848454f-jp65f\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:05.716091 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.716060 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-credential-socket\") pod \"e2e-inf-a-b48493-openshift-default-64848454f-jp65f\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:05.716470 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.716111 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istio-envoy\") pod \"e2e-inf-a-b48493-openshift-default-64848454f-jp65f\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:05.716470 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.716142 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istiod-ca-cert\") pod \"e2e-inf-a-b48493-openshift-default-64848454f-jp65f\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:05.716470 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.716357 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-workload-certs\") pod \"e2e-inf-a-b48493-openshift-default-64848454f-jp65f\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:05.716628 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.716525 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istio-data\") pod \"e2e-inf-a-b48493-openshift-default-64848454f-jp65f\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:05.716628 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.716613 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-workload-socket\") pod \"e2e-inf-a-b48493-openshift-default-64848454f-jp65f\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:05.716797 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.716775 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-credential-socket\") pod \"e2e-inf-a-b48493-openshift-default-64848454f-jp65f\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:05.716916 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.716895 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istiod-ca-cert\") pod \"e2e-inf-a-b48493-openshift-default-64848454f-jp65f\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:05.718886 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.718862 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istio-podinfo\") pod \"e2e-inf-a-b48493-openshift-default-64848454f-jp65f\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:05.718993 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.718926 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istio-envoy\") pod \"e2e-inf-a-b48493-openshift-default-64848454f-jp65f\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:05.725316 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.725280 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istio-token\") pod \"e2e-inf-a-b48493-openshift-default-64848454f-jp65f\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:05.725650 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.725631 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmlvt\" (UniqueName: \"kubernetes.io/projected/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-kube-api-access-dmlvt\") pod \"e2e-inf-a-b48493-openshift-default-64848454f-jp65f\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:05.833868 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.833765 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:05.994429 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.994399 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f"] Jul 07 00:21:05.996179 ip-10-0-136-25 kubenswrapper[2553]: W0707 00:21:05.996154 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ee1e932_37f9_4c5c_8abe_fc9d0c0df91e.slice/crio-95d24b07abbd9c594dd667544063f3e1c8d449af88f7c25c9eb172022e931e0b WatchSource:0}: Error finding container 95d24b07abbd9c594dd667544063f3e1c8d449af88f7c25c9eb172022e931e0b: Status 404 returned error can't find the container with id 95d24b07abbd9c594dd667544063f3e1c8d449af88f7c25c9eb172022e931e0b Jul 07 00:21:05.998816 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.998783 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:21:05.998919 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.998849 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:21:05.998919 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:05.998876 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:21:06.410361 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:06.410316 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" event={"ID":"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e","Type":"ContainerStarted","Data":"8cf953945b8334c54b77d9e155f36e7dca9f18b8f248fcd3eff5a54441c835e1"} Jul 07 00:21:06.410588 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:06.410378 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" event={"ID":"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e","Type":"ContainerStarted","Data":"95d24b07abbd9c594dd667544063f3e1c8d449af88f7c25c9eb172022e931e0b"} Jul 07 00:21:06.434268 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:06.434213 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" podStartSLOduration=1.4341951019999999 podStartE2EDuration="1.434195102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:21:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:21:06.431977497 +0000 UTC m=+1948.641707013" watchObservedRunningTime="2026-07-07 00:21:06.434195102 +0000 UTC m=+1948.643924688" Jul 07 00:21:06.834429 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:06.834284 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:07.840047 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:07.840014 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:08.422003 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:08.421970 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:08.423041 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:08.423019 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:13.643075 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.643034 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn"] Jul 07 00:21:13.643574 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.643559 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a11b345-a320-454f-83c8-a634bb7d78b4" containerName="istio-proxy" Jul 07 00:21:13.646817 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.646791 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:13.649990 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.649963 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-b8c86c-openshift-default-dockercfg-99p2n\"" Jul 07 00:21:13.662927 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.662899 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn"] Jul 07 00:21:13.801026 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.800988 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istio-data\") pod \"e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:13.801026 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.801023 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istio-podinfo\") pod \"e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:13.801335 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.801056 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-credential-socket\") pod \"e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:13.801335 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.801112 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istiod-ca-cert\") pod \"e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:13.801335 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.801208 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-workload-socket\") pod \"e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:13.801335 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.801244 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzndz\" (UniqueName: \"kubernetes.io/projected/d529b692-3c96-4ff7-92e5-dd8f47de4d68-kube-api-access-qzndz\") pod \"e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:13.801335 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.801279 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-workload-certs\") pod \"e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:13.801555 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.801341 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istio-token\") pod \"e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:13.801555 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.801380 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istio-envoy\") pod \"e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:13.902625 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.902518 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istio-token\") pod \"e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:13.902625 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.902565 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istio-envoy\") pod \"e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:13.902873 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.902645 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istio-data\") pod \"e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:13.902873 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.902671 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istio-podinfo\") pod \"e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:13.902873 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.902704 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-credential-socket\") pod \"e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:13.902873 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.902735 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istiod-ca-cert\") pod \"e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:13.902873 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.902794 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-workload-socket\") pod \"e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:13.902873 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.902831 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzndz\" (UniqueName: \"kubernetes.io/projected/d529b692-3c96-4ff7-92e5-dd8f47de4d68-kube-api-access-qzndz\") pod \"e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:13.903173 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.902932 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-workload-certs\") pod \"e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:13.903173 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.903153 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-credential-socket\") pod \"e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:13.903328 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.903283 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istio-data\") pod \"e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:13.903396 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.903371 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-workload-certs\") pod \"e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:13.903471 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.903448 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-workload-socket\") pod \"e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:13.903564 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.903548 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istiod-ca-cert\") pod \"e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:13.905088 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.905065 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istio-envoy\") pod \"e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:13.905343 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.905321 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istio-podinfo\") pod \"e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:13.910482 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.910458 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istio-token\") pod \"e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:13.910911 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.910891 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzndz\" (UniqueName: \"kubernetes.io/projected/d529b692-3c96-4ff7-92e5-dd8f47de4d68-kube-api-access-qzndz\") pod \"e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:13.959421 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:13.959381 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:14.119819 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:14.119785 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn"] Jul 07 00:21:14.121287 ip-10-0-136-25 kubenswrapper[2553]: W0707 00:21:14.121252 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd529b692_3c96_4ff7_92e5_dd8f47de4d68.slice/crio-814daf31b54643f3c480d4cf55f97b043e85cae7c9dcdf53a12408a8ba6d665d WatchSource:0}: Error finding container 814daf31b54643f3c480d4cf55f97b043e85cae7c9dcdf53a12408a8ba6d665d: Status 404 returned error can't find the container with id 814daf31b54643f3c480d4cf55f97b043e85cae7c9dcdf53a12408a8ba6d665d Jul 07 00:21:14.123833 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:14.123798 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:21:14.123949 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:14.123871 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:21:14.123949 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:14.123904 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:21:14.458441 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:14.458338 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" event={"ID":"d529b692-3c96-4ff7-92e5-dd8f47de4d68","Type":"ContainerStarted","Data":"9a7b84f7ff0245c4a44a0a11e0fe4a1953179cf13a57619f0781df324caa5ae9"} Jul 07 00:21:14.458441 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:14.458438 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" event={"ID":"d529b692-3c96-4ff7-92e5-dd8f47de4d68","Type":"ContainerStarted","Data":"814daf31b54643f3c480d4cf55f97b043e85cae7c9dcdf53a12408a8ba6d665d"} Jul 07 00:21:14.482482 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:14.482416 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" podStartSLOduration=1.482340632 podStartE2EDuration="1.482340632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:21:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:21:14.48077199 +0000 UTC m=+1956.690501532" watchObservedRunningTime="2026-07-07 00:21:14.482340632 +0000 UTC m=+1956.692070197" Jul 07 00:21:14.959685 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:14.959644 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:15.965604 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:15.965548 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:16.471442 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:16.471408 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:16.472275 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:16.472250 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:24.463531 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:24.463481 2553 patch_prober.go:28] interesting pod/e2e-inf-a-b48493-openshift-default-64848454f-jp65f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 00:21:24.464027 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:24.463559 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" podUID="5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 00:21:25.531159 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:25.531122 2553 generic.go:356] "Generic (PLEG): container finished" podID="5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e" containerID="8cf953945b8334c54b77d9e155f36e7dca9f18b8f248fcd3eff5a54441c835e1" exitCode=0 Jul 07 00:21:25.531653 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:25.531195 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" event={"ID":"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e","Type":"ContainerDied","Data":"8cf953945b8334c54b77d9e155f36e7dca9f18b8f248fcd3eff5a54441c835e1"} Jul 07 00:21:25.531707 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:25.531655 2553 scope.go:117] "RemoveContainer" containerID="8cf953945b8334c54b77d9e155f36e7dca9f18b8f248fcd3eff5a54441c835e1" Jul 07 00:21:25.532728 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:25.532703 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:21:25.532790 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:25.532779 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:21:25.532826 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:25.532817 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:21:26.544447 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:26.543369 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" event={"ID":"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e","Type":"ContainerStarted","Data":"b44c02f389e089f5e6b2774f2bd9f868af4100256261f39ef31504b472dc8ed9"} Jul 07 00:21:26.834968 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:26.834835 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:26.835511 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:26.835475 2553 patch_prober.go:28] interesting pod/e2e-inf-a-b48493-openshift-default-64848454f-jp65f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Jul 07 00:21:26.835648 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:26.835569 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" podUID="5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Jul 07 00:21:27.835436 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:27.834887 2553 patch_prober.go:28] interesting pod/e2e-inf-a-b48493-openshift-default-64848454f-jp65f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Jul 07 00:21:27.835436 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:27.834961 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" podUID="5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Jul 07 00:21:28.835360 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:28.835312 2553 patch_prober.go:28] interesting pod/e2e-inf-a-b48493-openshift-default-64848454f-jp65f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Jul 07 00:21:28.835592 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:28.835386 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" podUID="5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Jul 07 00:21:29.835235 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:29.835188 2553 patch_prober.go:28] interesting pod/e2e-inf-a-b48493-openshift-default-64848454f-jp65f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Jul 07 00:21:29.836037 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:29.835588 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" podUID="5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Jul 07 00:21:30.835614 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:30.835011 2553 patch_prober.go:28] interesting pod/e2e-inf-a-b48493-openshift-default-64848454f-jp65f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Jul 07 00:21:30.835614 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:30.835080 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" podUID="5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Jul 07 00:21:31.837911 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:31.837876 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:31.838437 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:31.838147 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:31.838965 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:31.838943 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:21:32.513519 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:32.513478 2553 patch_prober.go:28] interesting pod/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 00:21:32.513706 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:32.513548 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" podUID="d529b692-3c96-4ff7-92e5-dd8f47de4d68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 00:21:33.621278 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:33.621175 2553 generic.go:356] "Generic (PLEG): container finished" podID="d529b692-3c96-4ff7-92e5-dd8f47de4d68" containerID="9a7b84f7ff0245c4a44a0a11e0fe4a1953179cf13a57619f0781df324caa5ae9" exitCode=0 Jul 07 00:21:33.621278 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:33.621262 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" event={"ID":"d529b692-3c96-4ff7-92e5-dd8f47de4d68","Type":"ContainerDied","Data":"9a7b84f7ff0245c4a44a0a11e0fe4a1953179cf13a57619f0781df324caa5ae9"} Jul 07 00:21:33.621861 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:33.621841 2553 scope.go:117] "RemoveContainer" containerID="9a7b84f7ff0245c4a44a0a11e0fe4a1953179cf13a57619f0781df324caa5ae9" Jul 07 00:21:33.623076 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:33.623034 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:21:33.623190 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:33.623114 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:21:33.623190 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:33.623149 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:21:34.633513 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:34.633470 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" event={"ID":"d529b692-3c96-4ff7-92e5-dd8f47de4d68","Type":"ContainerStarted","Data":"fe4f071e0fd16dc96c86bd58f19667d5560f60ee94b8672e116aa0a2fc6a660e"} Jul 07 00:21:34.959892 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:34.959855 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:34.960090 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:34.960062 2553 patch_prober.go:28] interesting pod/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Jul 07 00:21:34.960142 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:34.960114 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" podUID="d529b692-3c96-4ff7-92e5-dd8f47de4d68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Jul 07 00:21:35.960440 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:35.960402 2553 patch_prober.go:28] interesting pod/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Jul 07 00:21:35.960868 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:35.960462 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" podUID="d529b692-3c96-4ff7-92e5-dd8f47de4d68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Jul 07 00:21:36.960705 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:36.960661 2553 patch_prober.go:28] interesting pod/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Jul 07 00:21:36.961275 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:36.960721 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" podUID="d529b692-3c96-4ff7-92e5-dd8f47de4d68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Jul 07 00:21:37.960719 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:37.960674 2553 patch_prober.go:28] interesting pod/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Jul 07 00:21:37.961218 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:37.960740 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" podUID="d529b692-3c96-4ff7-92e5-dd8f47de4d68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Jul 07 00:21:38.963716 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:38.963682 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:38.964203 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:38.964049 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:21:38.964710 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:21:38.964689 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:23:31.152940 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:31.152906 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f"] Jul 07 00:23:31.153483 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:31.153120 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" podUID="5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e" containerName="istio-proxy" containerID="cri-o://b44c02f389e089f5e6b2774f2bd9f868af4100256261f39ef31504b472dc8ed9" gracePeriod=30 Jul 07 00:23:31.841182 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:31.841148 2553 patch_prober.go:28] interesting pod/e2e-inf-a-b48493-openshift-default-64848454f-jp65f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 00:23:31.841385 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:31.841224 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" podUID="5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 00:23:36.383138 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.383105 2553 generic.go:356] "Generic (PLEG): container finished" podID="5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e" containerID="b44c02f389e089f5e6b2774f2bd9f868af4100256261f39ef31504b472dc8ed9" exitCode=0 Jul 07 00:23:36.383563 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.383172 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" event={"ID":"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e","Type":"ContainerDied","Data":"b44c02f389e089f5e6b2774f2bd9f868af4100256261f39ef31504b472dc8ed9"} Jul 07 00:23:36.383563 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.383236 2553 scope.go:117] "RemoveContainer" containerID="8cf953945b8334c54b77d9e155f36e7dca9f18b8f248fcd3eff5a54441c835e1" Jul 07 00:23:36.426827 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.426745 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:23:36.581223 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.581164 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istiod-ca-cert\") pod \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " Jul 07 00:23:36.581223 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.581227 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istio-data\") pod \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " Jul 07 00:23:36.581527 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.581253 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-workload-certs\") pod \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " Jul 07 00:23:36.581527 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.581382 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istio-envoy\") pod \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " Jul 07 00:23:36.581527 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.581414 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istio-token\") pod \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " Jul 07 00:23:36.581527 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.581432 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-credential-socket\") pod \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " Jul 07 00:23:36.581527 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.581452 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istio-podinfo\") pod \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " Jul 07 00:23:36.581527 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.581481 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmlvt\" (UniqueName: \"kubernetes.io/projected/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-kube-api-access-dmlvt\") pod \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " Jul 07 00:23:36.581527 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.581527 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-workload-socket\") pod \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\" (UID: \"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e\") " Jul 07 00:23:36.581878 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.581664 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e" (UID: "5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:23:36.581878 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.581685 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e" (UID: "5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:23:36.581878 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.581864 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e" (UID: "5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:23:36.582048 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.582002 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e" (UID: "5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:23:36.582109 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.582045 2553 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istiod-ca-cert\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:23:36.582109 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.582066 2553 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-workload-certs\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:23:36.582109 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.582081 2553 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-workload-socket\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:23:36.582350 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.582327 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istio-data" (OuterVolumeSpecName: "istio-data") pod "5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e" (UID: "5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:23:36.583938 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.583899 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e" (UID: "5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 00:23:36.584054 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.583962 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istio-token" (OuterVolumeSpecName: "istio-token") pod "5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e" (UID: "5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:23:36.584254 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.584231 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-kube-api-access-dmlvt" (OuterVolumeSpecName: "kube-api-access-dmlvt") pod "5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e" (UID: "5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e"). InnerVolumeSpecName "kube-api-access-dmlvt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:23:36.584347 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.584245 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e" (UID: "5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:23:36.683453 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.683342 2553 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istio-envoy\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:23:36.683453 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.683391 2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istio-token\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:23:36.683453 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.683402 2553 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-credential-socket\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:23:36.683453 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.683412 2553 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istio-podinfo\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:23:36.683453 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.683421 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmlvt\" (UniqueName: \"kubernetes.io/projected/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-kube-api-access-dmlvt\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:23:36.683453 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:36.683430 2553 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e-istio-data\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:23:37.390522 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:37.390489 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" Jul 07 00:23:37.391011 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:37.390530 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f" event={"ID":"5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e","Type":"ContainerDied","Data":"95d24b07abbd9c594dd667544063f3e1c8d449af88f7c25c9eb172022e931e0b"} Jul 07 00:23:37.391011 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:37.390584 2553 scope.go:117] "RemoveContainer" containerID="b44c02f389e089f5e6b2774f2bd9f868af4100256261f39ef31504b472dc8ed9" Jul 07 00:23:37.415263 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:37.415224 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f"] Jul 07 00:23:37.418081 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:37.418052 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-b48493-openshift-default-64848454f-jp65f"] Jul 07 00:23:38.303950 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:38.303912 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e" path="/var/lib/kubelet/pods/5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e/volumes" Jul 07 00:23:38.368913 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:38.368878 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn"] Jul 07 00:23:38.369125 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:38.369099 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" podUID="d529b692-3c96-4ff7-92e5-dd8f47de4d68" containerName="istio-proxy" containerID="cri-o://fe4f071e0fd16dc96c86bd58f19667d5560f60ee94b8672e116aa0a2fc6a660e" gracePeriod=30 Jul 07 00:23:38.418596 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:38.418563 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4z8vq_3253bdfe-5248-4d4c-9864-11660eb042ca/ovn-acl-logging/0.log" Jul 07 00:23:38.422167 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:38.422139 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4z8vq_3253bdfe-5248-4d4c-9864-11660eb042ca/ovn-acl-logging/0.log" Jul 07 00:23:38.966388 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:38.966347 2553 patch_prober.go:28] interesting pod/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 00:23:38.966571 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:38.966419 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" podUID="d529b692-3c96-4ff7-92e5-dd8f47de4d68" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 00:23:39.305834 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.305743 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t"] Jul 07 00:23:39.306222 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.306208 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e" containerName="istio-proxy" Jul 07 00:23:39.306264 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.306226 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e" containerName="istio-proxy" Jul 07 00:23:39.306264 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.306248 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e" containerName="istio-proxy" Jul 07 00:23:39.306264 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.306254 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e" containerName="istio-proxy" Jul 07 00:23:39.306393 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.306340 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e" containerName="istio-proxy" Jul 07 00:23:39.310879 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.310859 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:39.314554 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.314532 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-2833a6-openshift-default-dockercfg-6brcb\"" Jul 07 00:23:39.325596 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.325564 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t"] Jul 07 00:23:39.409604 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.409563 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-istio-data\") pod \"e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:39.409864 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.409672 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-workload-socket\") pod \"e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:39.409864 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.409704 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9zx8\" (UniqueName: \"kubernetes.io/projected/d690dd3b-e727-45e1-8053-0fa81d67574f-kube-api-access-k9zx8\") pod \"e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:39.409864 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.409813 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d690dd3b-e727-45e1-8053-0fa81d67574f-istiod-ca-cert\") pod \"e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:39.410146 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.409880 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d690dd3b-e727-45e1-8053-0fa81d67574f-istio-podinfo\") pod \"e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:39.410146 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.410036 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-credential-socket\") pod \"e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:39.410146 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.410066 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d690dd3b-e727-45e1-8053-0fa81d67574f-istio-token\") pod \"e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:39.410146 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.410099 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-workload-certs\") pod \"e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:39.410416 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.410159 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-istio-envoy\") pod \"e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:39.511603 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.511564 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d690dd3b-e727-45e1-8053-0fa81d67574f-istiod-ca-cert\") pod \"e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:39.512034 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.511613 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d690dd3b-e727-45e1-8053-0fa81d67574f-istio-podinfo\") pod \"e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:39.512034 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.511675 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-credential-socket\") pod \"e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:39.512034 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.511691 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d690dd3b-e727-45e1-8053-0fa81d67574f-istio-token\") pod \"e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:39.512034 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.511720 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-workload-certs\") pod \"e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:39.512034 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.511745 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-istio-envoy\") pod \"e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:39.512034 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.511782 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-istio-data\") pod \"e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:39.512034 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.511836 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-workload-socket\") pod \"e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:39.512034 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.511875 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9zx8\" (UniqueName: \"kubernetes.io/projected/d690dd3b-e727-45e1-8053-0fa81d67574f-kube-api-access-k9zx8\") pod \"e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:39.512434 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.512097 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-workload-certs\") pod \"e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:39.512434 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.512160 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-credential-socket\") pod \"e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:39.512434 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.512280 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-istio-data\") pod \"e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:39.512434 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.512406 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-workload-socket\") pod \"e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:39.512434 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.512412 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d690dd3b-e727-45e1-8053-0fa81d67574f-istiod-ca-cert\") pod \"e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:39.513996 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.513975 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-istio-envoy\") pod \"e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:39.514220 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.514202 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d690dd3b-e727-45e1-8053-0fa81d67574f-istio-podinfo\") pod \"e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:39.521449 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.521421 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9zx8\" (UniqueName: \"kubernetes.io/projected/d690dd3b-e727-45e1-8053-0fa81d67574f-kube-api-access-k9zx8\") pod \"e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:39.521703 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.521688 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d690dd3b-e727-45e1-8053-0fa81d67574f-istio-token\") pod \"e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:39.624631 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.624535 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:39.782954 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.782924 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t"] Jul 07 00:23:39.785267 ip-10-0-136-25 kubenswrapper[2553]: W0707 00:23:39.785244 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd690dd3b_e727_45e1_8053_0fa81d67574f.slice/crio-d4ae8f686d090507698cb2ab3d9472a89cbdf6e23125cc17a55e36162753b096 WatchSource:0}: Error finding container d4ae8f686d090507698cb2ab3d9472a89cbdf6e23125cc17a55e36162753b096: Status 404 returned error can't find the container with id d4ae8f686d090507698cb2ab3d9472a89cbdf6e23125cc17a55e36162753b096 Jul 07 00:23:39.787371 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.787335 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:23:39.787469 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.787418 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:23:39.787469 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:39.787452 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:23:40.413863 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:40.413827 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" event={"ID":"d690dd3b-e727-45e1-8053-0fa81d67574f","Type":"ContainerStarted","Data":"dbda6d978c9966e124154ca9f58e5264081eed907f0bc6ccdc4dd45c7bd28097"} Jul 07 00:23:40.414098 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:40.413890 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" event={"ID":"d690dd3b-e727-45e1-8053-0fa81d67574f","Type":"ContainerStarted","Data":"d4ae8f686d090507698cb2ab3d9472a89cbdf6e23125cc17a55e36162753b096"} Jul 07 00:23:40.438813 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:40.438751 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" podStartSLOduration=1.438725535 podStartE2EDuration="1.438725535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:23:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:23:40.436525264 +0000 UTC m=+2102.646254816" watchObservedRunningTime="2026-07-07 00:23:40.438725535 +0000 UTC m=+2102.648455058" Jul 07 00:23:40.624932 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:40.624889 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:40.630396 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:40.630365 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:41.421857 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:41.421819 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:41.422734 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:41.422710 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:43.640639 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:43.640573 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:23:43.659555 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:43.659523 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-workload-certs\") pod \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " Jul 07 00:23:43.659555 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:43.659561 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istio-token\") pod \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " Jul 07 00:23:43.659806 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:43.659591 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istio-envoy\") pod \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " Jul 07 00:23:43.659806 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:43.659634 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istiod-ca-cert\") pod \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " Jul 07 00:23:43.659806 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:43.659672 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qzndz\" (UniqueName: \"kubernetes.io/projected/d529b692-3c96-4ff7-92e5-dd8f47de4d68-kube-api-access-qzndz\") pod \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " Jul 07 00:23:43.659806 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:43.659724 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-credential-socket\") pod \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " Jul 07 00:23:43.659806 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:43.659785 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istio-data\") pod \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " Jul 07 00:23:43.660142 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:43.659835 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istio-podinfo\") pod \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " Jul 07 00:23:43.660142 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:43.659841 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d529b692-3c96-4ff7-92e5-dd8f47de4d68" (UID: "d529b692-3c96-4ff7-92e5-dd8f47de4d68"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:23:43.660142 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:43.659880 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-workload-socket\") pod \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\" (UID: \"d529b692-3c96-4ff7-92e5-dd8f47de4d68\") " Jul 07 00:23:43.660360 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:43.660137 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d529b692-3c96-4ff7-92e5-dd8f47de4d68" (UID: "d529b692-3c96-4ff7-92e5-dd8f47de4d68"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:23:43.660360 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:43.660171 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d529b692-3c96-4ff7-92e5-dd8f47de4d68" (UID: "d529b692-3c96-4ff7-92e5-dd8f47de4d68"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:23:43.660360 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:43.660282 2553 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-workload-certs\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:23:43.660360 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:43.660350 2553 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istiod-ca-cert\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:23:43.660595 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:43.660364 2553 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-credential-socket\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:23:43.660595 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:43.660399 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d529b692-3c96-4ff7-92e5-dd8f47de4d68" (UID: "d529b692-3c96-4ff7-92e5-dd8f47de4d68"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:23:43.660690 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:43.660660 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istio-data" (OuterVolumeSpecName: "istio-data") pod "d529b692-3c96-4ff7-92e5-dd8f47de4d68" (UID: "d529b692-3c96-4ff7-92e5-dd8f47de4d68"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:23:43.662165 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:43.662131 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d529b692-3c96-4ff7-92e5-dd8f47de4d68-kube-api-access-qzndz" (OuterVolumeSpecName: "kube-api-access-qzndz") pod "d529b692-3c96-4ff7-92e5-dd8f47de4d68" (UID: "d529b692-3c96-4ff7-92e5-dd8f47de4d68"). InnerVolumeSpecName "kube-api-access-qzndz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:23:43.662268 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:43.662179 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istio-token" (OuterVolumeSpecName: "istio-token") pod "d529b692-3c96-4ff7-92e5-dd8f47de4d68" (UID: "d529b692-3c96-4ff7-92e5-dd8f47de4d68"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:23:43.662344 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:43.662324 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d529b692-3c96-4ff7-92e5-dd8f47de4d68" (UID: "d529b692-3c96-4ff7-92e5-dd8f47de4d68"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:23:43.662464 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:43.662438 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d529b692-3c96-4ff7-92e5-dd8f47de4d68" (UID: "d529b692-3c96-4ff7-92e5-dd8f47de4d68"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 00:23:43.761680 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:43.761640 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzndz\" (UniqueName: \"kubernetes.io/projected/d529b692-3c96-4ff7-92e5-dd8f47de4d68-kube-api-access-qzndz\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:23:43.761680 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:43.761673 2553 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istio-data\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:23:43.761680 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:43.761682 2553 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istio-podinfo\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:23:43.761680 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:43.761690 2553 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-workload-socket\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:23:43.761959 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:43.761701 2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istio-token\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:23:43.761959 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:43.761709 2553 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d529b692-3c96-4ff7-92e5-dd8f47de4d68-istio-envoy\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:23:44.445164 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:44.445125 2553 generic.go:356] "Generic (PLEG): container finished" podID="d529b692-3c96-4ff7-92e5-dd8f47de4d68" containerID="fe4f071e0fd16dc96c86bd58f19667d5560f60ee94b8672e116aa0a2fc6a660e" exitCode=0 Jul 07 00:23:44.445369 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:44.445174 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" Jul 07 00:23:44.445369 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:44.445217 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" event={"ID":"d529b692-3c96-4ff7-92e5-dd8f47de4d68","Type":"ContainerDied","Data":"fe4f071e0fd16dc96c86bd58f19667d5560f60ee94b8672e116aa0a2fc6a660e"} Jul 07 00:23:44.445369 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:44.445270 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn" event={"ID":"d529b692-3c96-4ff7-92e5-dd8f47de4d68","Type":"ContainerDied","Data":"814daf31b54643f3c480d4cf55f97b043e85cae7c9dcdf53a12408a8ba6d665d"} Jul 07 00:23:44.445369 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:44.445319 2553 scope.go:117] "RemoveContainer" containerID="fe4f071e0fd16dc96c86bd58f19667d5560f60ee94b8672e116aa0a2fc6a660e" Jul 07 00:23:44.454847 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:44.454825 2553 scope.go:117] "RemoveContainer" containerID="9a7b84f7ff0245c4a44a0a11e0fe4a1953179cf13a57619f0781df324caa5ae9" Jul 07 00:23:44.463705 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:44.463681 2553 scope.go:117] "RemoveContainer" containerID="fe4f071e0fd16dc96c86bd58f19667d5560f60ee94b8672e116aa0a2fc6a660e" Jul 07 00:23:44.464013 ip-10-0-136-25 kubenswrapper[2553]: E0707 00:23:44.463993 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe4f071e0fd16dc96c86bd58f19667d5560f60ee94b8672e116aa0a2fc6a660e\": container with ID starting with fe4f071e0fd16dc96c86bd58f19667d5560f60ee94b8672e116aa0a2fc6a660e not found: ID does not exist" containerID="fe4f071e0fd16dc96c86bd58f19667d5560f60ee94b8672e116aa0a2fc6a660e" Jul 07 00:23:44.464121 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:44.464029 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe4f071e0fd16dc96c86bd58f19667d5560f60ee94b8672e116aa0a2fc6a660e"} err="failed to get container status \"fe4f071e0fd16dc96c86bd58f19667d5560f60ee94b8672e116aa0a2fc6a660e\": rpc error: code = NotFound desc = could not find container \"fe4f071e0fd16dc96c86bd58f19667d5560f60ee94b8672e116aa0a2fc6a660e\": container with ID starting with fe4f071e0fd16dc96c86bd58f19667d5560f60ee94b8672e116aa0a2fc6a660e not found: ID does not exist" Jul 07 00:23:44.464121 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:44.464054 2553 scope.go:117] "RemoveContainer" containerID="9a7b84f7ff0245c4a44a0a11e0fe4a1953179cf13a57619f0781df324caa5ae9" Jul 07 00:23:44.464371 ip-10-0-136-25 kubenswrapper[2553]: E0707 00:23:44.464351 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a7b84f7ff0245c4a44a0a11e0fe4a1953179cf13a57619f0781df324caa5ae9\": container with ID starting with 9a7b84f7ff0245c4a44a0a11e0fe4a1953179cf13a57619f0781df324caa5ae9 not found: ID does not exist" containerID="9a7b84f7ff0245c4a44a0a11e0fe4a1953179cf13a57619f0781df324caa5ae9" Jul 07 00:23:44.464419 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:44.464378 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a7b84f7ff0245c4a44a0a11e0fe4a1953179cf13a57619f0781df324caa5ae9"} err="failed to get container status \"9a7b84f7ff0245c4a44a0a11e0fe4a1953179cf13a57619f0781df324caa5ae9\": rpc error: code = NotFound desc = could not find container \"9a7b84f7ff0245c4a44a0a11e0fe4a1953179cf13a57619f0781df324caa5ae9\": container with ID starting with 9a7b84f7ff0245c4a44a0a11e0fe4a1953179cf13a57619f0781df324caa5ae9 not found: ID does not exist" Jul 07 00:23:44.479643 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:44.479606 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn"] Jul 07 00:23:44.484176 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:44.484143 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-b8c86c-openshift-default-6767bfd47-z4qbn"] Jul 07 00:23:46.303408 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:46.303372 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d529b692-3c96-4ff7-92e5-dd8f47de4d68" path="/var/lib/kubelet/pods/d529b692-3c96-4ff7-92e5-dd8f47de4d68/volumes" Jul 07 00:23:47.752488 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.742822 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh"] Jul 07 00:23:47.753734 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.753704 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d529b692-3c96-4ff7-92e5-dd8f47de4d68" containerName="istio-proxy" Jul 07 00:23:47.753902 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.753890 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="d529b692-3c96-4ff7-92e5-dd8f47de4d68" containerName="istio-proxy" Jul 07 00:23:47.754160 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.754145 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="d529b692-3c96-4ff7-92e5-dd8f47de4d68" containerName="istio-proxy" Jul 07 00:23:47.754260 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.754250 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ee1e932-37f9-4c5c-8abe-fc9d0c0df91e" containerName="istio-proxy" Jul 07 00:23:47.754383 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.754370 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="d529b692-3c96-4ff7-92e5-dd8f47de4d68" containerName="istio-proxy" Jul 07 00:23:47.759916 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.759888 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:47.762911 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.762880 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-28187d-openshift-default-dockercfg-mnzqc\"" Jul 07 00:23:47.763154 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.763132 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh"] Jul 07 00:23:47.803979 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.803939 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-workload-socket\") pod \"e2e-api-b-28187d-openshift-default-6b49b767df-92kbh\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:47.804211 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.803986 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68cc7c09-ac3d-464b-8542-e1cc8000610c-istio-podinfo\") pod \"e2e-api-b-28187d-openshift-default-6b49b767df-92kbh\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:47.804211 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.804016 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-istio-data\") pod \"e2e-api-b-28187d-openshift-default-6b49b767df-92kbh\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:47.804211 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.804103 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-credential-socket\") pod \"e2e-api-b-28187d-openshift-default-6b49b767df-92kbh\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:47.804211 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.804149 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68cc7c09-ac3d-464b-8542-e1cc8000610c-istio-token\") pod \"e2e-api-b-28187d-openshift-default-6b49b767df-92kbh\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:47.804211 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.804193 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j69rb\" (UniqueName: \"kubernetes.io/projected/68cc7c09-ac3d-464b-8542-e1cc8000610c-kube-api-access-j69rb\") pod \"e2e-api-b-28187d-openshift-default-6b49b767df-92kbh\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:47.804464 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.804222 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68cc7c09-ac3d-464b-8542-e1cc8000610c-istiod-ca-cert\") pod \"e2e-api-b-28187d-openshift-default-6b49b767df-92kbh\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:47.804464 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.804344 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-workload-certs\") pod \"e2e-api-b-28187d-openshift-default-6b49b767df-92kbh\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:47.804464 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.804439 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-istio-envoy\") pod \"e2e-api-b-28187d-openshift-default-6b49b767df-92kbh\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:47.905800 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.905752 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-credential-socket\") pod \"e2e-api-b-28187d-openshift-default-6b49b767df-92kbh\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:47.906030 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.905816 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68cc7c09-ac3d-464b-8542-e1cc8000610c-istio-token\") pod \"e2e-api-b-28187d-openshift-default-6b49b767df-92kbh\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:47.906030 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.905846 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j69rb\" (UniqueName: \"kubernetes.io/projected/68cc7c09-ac3d-464b-8542-e1cc8000610c-kube-api-access-j69rb\") pod \"e2e-api-b-28187d-openshift-default-6b49b767df-92kbh\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:47.906030 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.905872 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68cc7c09-ac3d-464b-8542-e1cc8000610c-istiod-ca-cert\") pod \"e2e-api-b-28187d-openshift-default-6b49b767df-92kbh\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:47.906030 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.906014 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-workload-certs\") pod \"e2e-api-b-28187d-openshift-default-6b49b767df-92kbh\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:47.906261 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.906111 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-istio-envoy\") pod \"e2e-api-b-28187d-openshift-default-6b49b767df-92kbh\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:47.906261 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.906184 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-workload-socket\") pod \"e2e-api-b-28187d-openshift-default-6b49b767df-92kbh\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:47.906261 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.906215 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-credential-socket\") pod \"e2e-api-b-28187d-openshift-default-6b49b767df-92kbh\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:47.906261 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.906218 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68cc7c09-ac3d-464b-8542-e1cc8000610c-istio-podinfo\") pod \"e2e-api-b-28187d-openshift-default-6b49b767df-92kbh\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:47.906507 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.906276 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-istio-data\") pod \"e2e-api-b-28187d-openshift-default-6b49b767df-92kbh\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:47.906507 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.906468 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-workload-certs\") pod \"e2e-api-b-28187d-openshift-default-6b49b767df-92kbh\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:47.906592 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.906516 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-workload-socket\") pod \"e2e-api-b-28187d-openshift-default-6b49b767df-92kbh\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:47.906592 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.906560 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-istio-data\") pod \"e2e-api-b-28187d-openshift-default-6b49b767df-92kbh\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:47.906710 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.906689 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68cc7c09-ac3d-464b-8542-e1cc8000610c-istiod-ca-cert\") pod \"e2e-api-b-28187d-openshift-default-6b49b767df-92kbh\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:47.908576 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.908551 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-istio-envoy\") pod \"e2e-api-b-28187d-openshift-default-6b49b767df-92kbh\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:47.908783 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.908765 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68cc7c09-ac3d-464b-8542-e1cc8000610c-istio-podinfo\") pod \"e2e-api-b-28187d-openshift-default-6b49b767df-92kbh\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:47.914607 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.914582 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j69rb\" (UniqueName: \"kubernetes.io/projected/68cc7c09-ac3d-464b-8542-e1cc8000610c-kube-api-access-j69rb\") pod \"e2e-api-b-28187d-openshift-default-6b49b767df-92kbh\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:47.914956 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:47.914938 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68cc7c09-ac3d-464b-8542-e1cc8000610c-istio-token\") pod \"e2e-api-b-28187d-openshift-default-6b49b767df-92kbh\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:48.080780 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:48.080682 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:48.296042 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:48.296014 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh"] Jul 07 00:23:48.297739 ip-10-0-136-25 kubenswrapper[2553]: W0707 00:23:48.297708 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68cc7c09_ac3d_464b_8542_e1cc8000610c.slice/crio-0866b2ad567b042da19e2e92cdd5e294f27a178f2ce86d48cdab9c48ee0b55b8 WatchSource:0}: Error finding container 0866b2ad567b042da19e2e92cdd5e294f27a178f2ce86d48cdab9c48ee0b55b8: Status 404 returned error can't find the container with id 0866b2ad567b042da19e2e92cdd5e294f27a178f2ce86d48cdab9c48ee0b55b8 Jul 07 00:23:48.300193 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:48.300152 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:23:48.300319 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:48.300226 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:23:48.300319 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:48.300255 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:23:48.473820 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:48.473768 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" event={"ID":"68cc7c09-ac3d-464b-8542-e1cc8000610c","Type":"ContainerStarted","Data":"0ce2b46fdfc3a1a403628e77862ccdc3b2b1960eb2dbfc9d64c5a3b92d98d7c5"} Jul 07 00:23:48.474028 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:48.473842 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" event={"ID":"68cc7c09-ac3d-464b-8542-e1cc8000610c","Type":"ContainerStarted","Data":"0866b2ad567b042da19e2e92cdd5e294f27a178f2ce86d48cdab9c48ee0b55b8"} Jul 07 00:23:48.496343 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:48.496270 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" podStartSLOduration=1.496246512 podStartE2EDuration="1.496246512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:23:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:23:48.493231729 +0000 UTC m=+2110.702961273" watchObservedRunningTime="2026-07-07 00:23:48.496246512 +0000 UTC m=+2110.705976035" Jul 07 00:23:49.081583 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:49.081537 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:49.083255 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:49.083231 2553 patch_prober.go:28] interesting pod/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Jul 07 00:23:49.083337 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:49.083311 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" podUID="68cc7c09-ac3d-464b-8542-e1cc8000610c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Jul 07 00:23:50.086016 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:50.085983 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:50.485459 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:50.485423 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:50.486434 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:50.486411 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:23:54.512287 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:54.512248 2553 generic.go:356] "Generic (PLEG): container finished" podID="d690dd3b-e727-45e1-8053-0fa81d67574f" containerID="dbda6d978c9966e124154ca9f58e5264081eed907f0bc6ccdc4dd45c7bd28097" exitCode=0 Jul 07 00:23:54.512736 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:54.512334 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" event={"ID":"d690dd3b-e727-45e1-8053-0fa81d67574f","Type":"ContainerDied","Data":"dbda6d978c9966e124154ca9f58e5264081eed907f0bc6ccdc4dd45c7bd28097"} Jul 07 00:23:54.512782 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:54.512764 2553 scope.go:117] "RemoveContainer" containerID="dbda6d978c9966e124154ca9f58e5264081eed907f0bc6ccdc4dd45c7bd28097" Jul 07 00:23:54.513868 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:54.513844 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:23:54.513926 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:54.513907 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:23:54.513961 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:54.513941 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:23:55.522834 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:55.522791 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" event={"ID":"d690dd3b-e727-45e1-8053-0fa81d67574f","Type":"ContainerStarted","Data":"869b0cd30537c37e86b66cd235c3a90517634247d7be64b1cdfcf03a140f9129"} Jul 07 00:23:55.626169 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:55.626122 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:55.626420 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:55.626361 2553 patch_prober.go:28] interesting pod/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Jul 07 00:23:55.626499 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:55.626424 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" podUID="d690dd3b-e727-45e1-8053-0fa81d67574f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Jul 07 00:23:56.422111 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:56.422068 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:23:56.625760 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:56.625717 2553 patch_prober.go:28] interesting pod/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Jul 07 00:23:56.626270 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:56.625793 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" podUID="d690dd3b-e727-45e1-8053-0fa81d67574f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Jul 07 00:23:57.625903 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:57.625860 2553 patch_prober.go:28] interesting pod/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Jul 07 00:23:57.626603 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:57.626565 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" podUID="d690dd3b-e727-45e1-8053-0fa81d67574f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Jul 07 00:23:58.625654 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:58.625612 2553 patch_prober.go:28] interesting pod/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Jul 07 00:23:58.625973 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:58.625946 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" podUID="d690dd3b-e727-45e1-8053-0fa81d67574f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Jul 07 00:23:59.626281 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:59.626239 2553 patch_prober.go:28] interesting pod/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Jul 07 00:23:59.626977 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:23:59.626943 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" podUID="d690dd3b-e727-45e1-8053-0fa81d67574f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Jul 07 00:24:00.629682 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:00.629651 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:24:00.630905 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:00.630880 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:24:02.583143 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:02.583057 2553 generic.go:356] "Generic (PLEG): container finished" podID="68cc7c09-ac3d-464b-8542-e1cc8000610c" containerID="0ce2b46fdfc3a1a403628e77862ccdc3b2b1960eb2dbfc9d64c5a3b92d98d7c5" exitCode=0 Jul 07 00:24:02.583143 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:02.583131 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" event={"ID":"68cc7c09-ac3d-464b-8542-e1cc8000610c","Type":"ContainerDied","Data":"0ce2b46fdfc3a1a403628e77862ccdc3b2b1960eb2dbfc9d64c5a3b92d98d7c5"} Jul 07 00:24:02.583659 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:02.583566 2553 scope.go:117] "RemoveContainer" containerID="0ce2b46fdfc3a1a403628e77862ccdc3b2b1960eb2dbfc9d64c5a3b92d98d7c5" Jul 07 00:24:02.584617 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:02.584592 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:24:02.584681 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:02.584661 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:24:02.584727 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:02.584692 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:24:03.591149 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:03.591112 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" event={"ID":"68cc7c09-ac3d-464b-8542-e1cc8000610c","Type":"ContainerStarted","Data":"b0922abffa0061b8a312a11336102a3865e6cf158c5c5b47bd63907d234b4830"} Jul 07 00:24:04.081879 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:04.081841 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:24:04.083281 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:04.083241 2553 patch_prober.go:28] interesting pod/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Jul 07 00:24:04.083461 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:04.083368 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" podUID="68cc7c09-ac3d-464b-8542-e1cc8000610c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Jul 07 00:24:05.081345 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:05.081284 2553 patch_prober.go:28] interesting pod/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Jul 07 00:24:05.081779 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:05.081385 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" podUID="68cc7c09-ac3d-464b-8542-e1cc8000610c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Jul 07 00:24:05.485443 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:05.485410 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:24:06.081081 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:06.081044 2553 patch_prober.go:28] interesting pod/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Jul 07 00:24:06.081269 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:06.081120 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" podUID="68cc7c09-ac3d-464b-8542-e1cc8000610c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Jul 07 00:24:07.081450 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:07.081410 2553 patch_prober.go:28] interesting pod/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Jul 07 00:24:07.081988 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:07.081476 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" podUID="68cc7c09-ac3d-464b-8542-e1cc8000610c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Jul 07 00:24:08.085096 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:08.085061 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:24:08.085999 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:08.085976 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:24:09.897086 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:09.897049 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t"] Jul 07 00:24:09.897615 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:09.897260 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" podUID="d690dd3b-e727-45e1-8053-0fa81d67574f" containerName="istio-proxy" containerID="cri-o://869b0cd30537c37e86b66cd235c3a90517634247d7be64b1cdfcf03a140f9129" gracePeriod=30 Jul 07 00:24:15.190743 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.190716 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:24:15.212396 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.212353 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d690dd3b-e727-45e1-8053-0fa81d67574f-istiod-ca-cert\") pod \"d690dd3b-e727-45e1-8053-0fa81d67574f\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " Jul 07 00:24:15.212396 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.212398 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d690dd3b-e727-45e1-8053-0fa81d67574f-istio-podinfo\") pod \"d690dd3b-e727-45e1-8053-0fa81d67574f\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " Jul 07 00:24:15.212653 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.212435 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-istio-data\") pod \"d690dd3b-e727-45e1-8053-0fa81d67574f\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " Jul 07 00:24:15.212653 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.212515 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-workload-certs\") pod \"d690dd3b-e727-45e1-8053-0fa81d67574f\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " Jul 07 00:24:15.212653 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.212543 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9zx8\" (UniqueName: \"kubernetes.io/projected/d690dd3b-e727-45e1-8053-0fa81d67574f-kube-api-access-k9zx8\") pod \"d690dd3b-e727-45e1-8053-0fa81d67574f\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " Jul 07 00:24:15.212653 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.212587 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-istio-envoy\") pod \"d690dd3b-e727-45e1-8053-0fa81d67574f\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " Jul 07 00:24:15.212653 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.212649 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-workload-socket\") pod \"d690dd3b-e727-45e1-8053-0fa81d67574f\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " Jul 07 00:24:15.212912 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.212862 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d690dd3b-e727-45e1-8053-0fa81d67574f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d690dd3b-e727-45e1-8053-0fa81d67574f" (UID: "d690dd3b-e727-45e1-8053-0fa81d67574f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:24:15.213001 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.212935 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d690dd3b-e727-45e1-8053-0fa81d67574f" (UID: "d690dd3b-e727-45e1-8053-0fa81d67574f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:24:15.213001 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.212984 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-credential-socket\") pod \"d690dd3b-e727-45e1-8053-0fa81d67574f\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " Jul 07 00:24:15.213131 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.213049 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d690dd3b-e727-45e1-8053-0fa81d67574f-istio-token\") pod \"d690dd3b-e727-45e1-8053-0fa81d67574f\" (UID: \"d690dd3b-e727-45e1-8053-0fa81d67574f\") " Jul 07 00:24:15.213190 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.213168 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d690dd3b-e727-45e1-8053-0fa81d67574f" (UID: "d690dd3b-e727-45e1-8053-0fa81d67574f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:24:15.213616 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.213445 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d690dd3b-e727-45e1-8053-0fa81d67574f" (UID: "d690dd3b-e727-45e1-8053-0fa81d67574f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:24:15.213616 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.213463 2553 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d690dd3b-e727-45e1-8053-0fa81d67574f-istiod-ca-cert\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:24:15.213616 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.213482 2553 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-workload-certs\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:24:15.213616 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.213493 2553 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-credential-socket\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:24:15.213977 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.213932 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-istio-data" (OuterVolumeSpecName: "istio-data") pod "d690dd3b-e727-45e1-8053-0fa81d67574f" (UID: "d690dd3b-e727-45e1-8053-0fa81d67574f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:24:15.215911 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.215868 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d690dd3b-e727-45e1-8053-0fa81d67574f-kube-api-access-k9zx8" (OuterVolumeSpecName: "kube-api-access-k9zx8") pod "d690dd3b-e727-45e1-8053-0fa81d67574f" (UID: "d690dd3b-e727-45e1-8053-0fa81d67574f"). InnerVolumeSpecName "kube-api-access-k9zx8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:24:15.216140 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.216115 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d690dd3b-e727-45e1-8053-0fa81d67574f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d690dd3b-e727-45e1-8053-0fa81d67574f" (UID: "d690dd3b-e727-45e1-8053-0fa81d67574f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 00:24:15.216236 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.216165 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d690dd3b-e727-45e1-8053-0fa81d67574f-istio-token" (OuterVolumeSpecName: "istio-token") pod "d690dd3b-e727-45e1-8053-0fa81d67574f" (UID: "d690dd3b-e727-45e1-8053-0fa81d67574f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:24:15.217143 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.217120 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d690dd3b-e727-45e1-8053-0fa81d67574f" (UID: "d690dd3b-e727-45e1-8053-0fa81d67574f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:24:15.314851 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.314805 2553 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-workload-socket\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:24:15.314851 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.314845 2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d690dd3b-e727-45e1-8053-0fa81d67574f-istio-token\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:24:15.314851 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.314858 2553 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d690dd3b-e727-45e1-8053-0fa81d67574f-istio-podinfo\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:24:15.315116 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.314868 2553 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-istio-data\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:24:15.315116 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.314879 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9zx8\" (UniqueName: \"kubernetes.io/projected/d690dd3b-e727-45e1-8053-0fa81d67574f-kube-api-access-k9zx8\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:24:15.315116 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.314892 2553 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d690dd3b-e727-45e1-8053-0fa81d67574f-istio-envoy\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:24:15.678425 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.678393 2553 generic.go:356] "Generic (PLEG): container finished" podID="d690dd3b-e727-45e1-8053-0fa81d67574f" containerID="869b0cd30537c37e86b66cd235c3a90517634247d7be64b1cdfcf03a140f9129" exitCode=0 Jul 07 00:24:15.678636 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.678448 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" Jul 07 00:24:15.678636 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.678475 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" event={"ID":"d690dd3b-e727-45e1-8053-0fa81d67574f","Type":"ContainerDied","Data":"869b0cd30537c37e86b66cd235c3a90517634247d7be64b1cdfcf03a140f9129"} Jul 07 00:24:15.678636 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.678530 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t" event={"ID":"d690dd3b-e727-45e1-8053-0fa81d67574f","Type":"ContainerDied","Data":"d4ae8f686d090507698cb2ab3d9472a89cbdf6e23125cc17a55e36162753b096"} Jul 07 00:24:15.678636 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.678551 2553 scope.go:117] "RemoveContainer" containerID="869b0cd30537c37e86b66cd235c3a90517634247d7be64b1cdfcf03a140f9129" Jul 07 00:24:15.689003 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.688983 2553 scope.go:117] "RemoveContainer" containerID="dbda6d978c9966e124154ca9f58e5264081eed907f0bc6ccdc4dd45c7bd28097" Jul 07 00:24:15.698135 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.698115 2553 scope.go:117] "RemoveContainer" containerID="869b0cd30537c37e86b66cd235c3a90517634247d7be64b1cdfcf03a140f9129" Jul 07 00:24:15.698405 ip-10-0-136-25 kubenswrapper[2553]: E0707 00:24:15.698388 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"869b0cd30537c37e86b66cd235c3a90517634247d7be64b1cdfcf03a140f9129\": container with ID starting with 869b0cd30537c37e86b66cd235c3a90517634247d7be64b1cdfcf03a140f9129 not found: ID does not exist" containerID="869b0cd30537c37e86b66cd235c3a90517634247d7be64b1cdfcf03a140f9129" Jul 07 00:24:15.698473 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.698414 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"869b0cd30537c37e86b66cd235c3a90517634247d7be64b1cdfcf03a140f9129"} err="failed to get container status \"869b0cd30537c37e86b66cd235c3a90517634247d7be64b1cdfcf03a140f9129\": rpc error: code = NotFound desc = could not find container \"869b0cd30537c37e86b66cd235c3a90517634247d7be64b1cdfcf03a140f9129\": container with ID starting with 869b0cd30537c37e86b66cd235c3a90517634247d7be64b1cdfcf03a140f9129 not found: ID does not exist" Jul 07 00:24:15.698473 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.698432 2553 scope.go:117] "RemoveContainer" containerID="dbda6d978c9966e124154ca9f58e5264081eed907f0bc6ccdc4dd45c7bd28097" Jul 07 00:24:15.698661 ip-10-0-136-25 kubenswrapper[2553]: E0707 00:24:15.698645 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbda6d978c9966e124154ca9f58e5264081eed907f0bc6ccdc4dd45c7bd28097\": container with ID starting with dbda6d978c9966e124154ca9f58e5264081eed907f0bc6ccdc4dd45c7bd28097 not found: ID does not exist" containerID="dbda6d978c9966e124154ca9f58e5264081eed907f0bc6ccdc4dd45c7bd28097" Jul 07 00:24:15.698706 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.698664 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbda6d978c9966e124154ca9f58e5264081eed907f0bc6ccdc4dd45c7bd28097"} err="failed to get container status \"dbda6d978c9966e124154ca9f58e5264081eed907f0bc6ccdc4dd45c7bd28097\": rpc error: code = NotFound desc = could not find container \"dbda6d978c9966e124154ca9f58e5264081eed907f0bc6ccdc4dd45c7bd28097\": container with ID starting with dbda6d978c9966e124154ca9f58e5264081eed907f0bc6ccdc4dd45c7bd28097 not found: ID does not exist" Jul 07 00:24:15.703061 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.703032 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t"] Jul 07 00:24:15.709852 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:15.709828 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-2833a6-openshift-default-68fb484d89-bb74t"] Jul 07 00:24:16.305180 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:16.305144 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d690dd3b-e727-45e1-8053-0fa81d67574f" path="/var/lib/kubelet/pods/d690dd3b-e727-45e1-8053-0fa81d67574f/volumes" Jul 07 00:24:16.882470 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:16.882438 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh"] Jul 07 00:24:16.882681 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:16.882646 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" podUID="68cc7c09-ac3d-464b-8542-e1cc8000610c" containerName="istio-proxy" containerID="cri-o://b0922abffa0061b8a312a11336102a3865e6cf158c5c5b47bd63907d234b4830" gracePeriod=30 Jul 07 00:24:17.851386 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:17.851352 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77"] Jul 07 00:24:17.851984 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:17.851963 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d690dd3b-e727-45e1-8053-0fa81d67574f" containerName="istio-proxy" Jul 07 00:24:17.852098 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:17.851986 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="d690dd3b-e727-45e1-8053-0fa81d67574f" containerName="istio-proxy" Jul 07 00:24:17.852098 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:17.852032 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d529b692-3c96-4ff7-92e5-dd8f47de4d68" containerName="istio-proxy" Jul 07 00:24:17.852098 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:17.852042 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="d529b692-3c96-4ff7-92e5-dd8f47de4d68" containerName="istio-proxy" Jul 07 00:24:17.852249 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:17.852159 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="d690dd3b-e727-45e1-8053-0fa81d67574f" containerName="istio-proxy" Jul 07 00:24:17.856882 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:17.856858 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:17.859614 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:17.859591 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-fced7e-openshift-default-dockercfg-lh7gh\"" Jul 07 00:24:17.873140 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:17.873110 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77"] Jul 07 00:24:17.943259 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:17.943218 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4787a56a-3a04-45b1-91a9-c44a9883025d-istio-token\") pod \"e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:17.943581 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:17.943559 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-credential-socket\") pod \"e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:17.943746 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:17.943731 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-workload-certs\") pod \"e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:17.943874 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:17.943861 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-istio-data\") pod \"e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:17.943981 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:17.943970 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-workload-socket\") pod \"e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:17.944158 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:17.944121 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvv9z\" (UniqueName: \"kubernetes.io/projected/4787a56a-3a04-45b1-91a9-c44a9883025d-kube-api-access-jvv9z\") pod \"e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:17.944249 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:17.944214 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-istio-envoy\") pod \"e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:17.944330 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:17.944258 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4787a56a-3a04-45b1-91a9-c44a9883025d-istio-podinfo\") pod \"e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:17.944393 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:17.944326 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4787a56a-3a04-45b1-91a9-c44a9883025d-istiod-ca-cert\") pod \"e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:18.044997 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:18.044956 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4787a56a-3a04-45b1-91a9-c44a9883025d-istiod-ca-cert\") pod \"e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:18.045184 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:18.045035 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4787a56a-3a04-45b1-91a9-c44a9883025d-istio-token\") pod \"e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:18.045184 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:18.045140 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-credential-socket\") pod \"e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:18.045343 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:18.045223 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-workload-certs\") pod \"e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:18.045343 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:18.045264 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-istio-data\") pod \"e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:18.045343 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:18.045317 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-workload-socket\") pod \"e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:18.045343 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:18.045337 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvv9z\" (UniqueName: \"kubernetes.io/projected/4787a56a-3a04-45b1-91a9-c44a9883025d-kube-api-access-jvv9z\") pod \"e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:18.045557 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:18.045372 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-istio-envoy\") pod \"e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:18.045557 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:18.045403 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4787a56a-3a04-45b1-91a9-c44a9883025d-istio-podinfo\") pod \"e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:18.045662 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:18.045568 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-credential-socket\") pod \"e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:18.045719 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:18.045658 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-workload-certs\") pod \"e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:18.045764 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:18.045713 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-istio-data\") pod \"e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:18.045829 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:18.045813 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-workload-socket\") pod \"e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:18.045901 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:18.045875 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4787a56a-3a04-45b1-91a9-c44a9883025d-istiod-ca-cert\") pod \"e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:18.047632 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:18.047611 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-istio-envoy\") pod \"e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:18.047821 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:18.047805 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4787a56a-3a04-45b1-91a9-c44a9883025d-istio-podinfo\") pod \"e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:18.053897 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:18.053874 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4787a56a-3a04-45b1-91a9-c44a9883025d-istio-token\") pod \"e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:18.054128 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:18.054112 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvv9z\" (UniqueName: \"kubernetes.io/projected/4787a56a-3a04-45b1-91a9-c44a9883025d-kube-api-access-jvv9z\") pod \"e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:18.166982 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:18.166948 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:18.322508 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:18.322475 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77"] Jul 07 00:24:18.324772 ip-10-0-136-25 kubenswrapper[2553]: W0707 00:24:18.324738 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4787a56a_3a04_45b1_91a9_c44a9883025d.slice/crio-d8c4963b7e23848aa21f9930723e14a9a5098cf0c8e75420e592f752cad46ea9 WatchSource:0}: Error finding container d8c4963b7e23848aa21f9930723e14a9a5098cf0c8e75420e592f752cad46ea9: Status 404 returned error can't find the container with id d8c4963b7e23848aa21f9930723e14a9a5098cf0c8e75420e592f752cad46ea9 Jul 07 00:24:18.327237 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:18.327203 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:24:18.327354 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:18.327277 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:24:18.327354 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:18.327329 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:24:18.699911 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:18.699876 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" event={"ID":"4787a56a-3a04-45b1-91a9-c44a9883025d","Type":"ContainerStarted","Data":"ce585010bb3521df6906c87747043b03f97c3011ad5855bbc4659a16053cb203"} Jul 07 00:24:18.700135 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:18.699940 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" event={"ID":"4787a56a-3a04-45b1-91a9-c44a9883025d","Type":"ContainerStarted","Data":"d8c4963b7e23848aa21f9930723e14a9a5098cf0c8e75420e592f752cad46ea9"} Jul 07 00:24:18.721838 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:18.721781 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" podStartSLOduration=1.721763953 podStartE2EDuration="1.721763953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:24:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:24:18.719428124 +0000 UTC m=+2140.929157668" watchObservedRunningTime="2026-07-07 00:24:18.721763953 +0000 UTC m=+2140.931493474" Jul 07 00:24:19.167560 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:19.167521 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:19.172532 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:19.172505 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:19.706756 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:19.706724 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:19.707552 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:19.707504 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:22.144144 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.144114 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:24:22.189434 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.189396 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68cc7c09-ac3d-464b-8542-e1cc8000610c-istiod-ca-cert\") pod \"68cc7c09-ac3d-464b-8542-e1cc8000610c\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " Jul 07 00:24:22.189617 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.189474 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-workload-certs\") pod \"68cc7c09-ac3d-464b-8542-e1cc8000610c\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " Jul 07 00:24:22.189617 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.189503 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-workload-socket\") pod \"68cc7c09-ac3d-464b-8542-e1cc8000610c\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " Jul 07 00:24:22.189617 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.189559 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-istio-envoy\") pod \"68cc7c09-ac3d-464b-8542-e1cc8000610c\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " Jul 07 00:24:22.189617 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.189602 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68cc7c09-ac3d-464b-8542-e1cc8000610c-istio-podinfo\") pod \"68cc7c09-ac3d-464b-8542-e1cc8000610c\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " Jul 07 00:24:22.189840 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.189626 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-credential-socket\") pod \"68cc7c09-ac3d-464b-8542-e1cc8000610c\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " Jul 07 00:24:22.189840 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.189685 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68cc7c09-ac3d-464b-8542-e1cc8000610c-istio-token\") pod \"68cc7c09-ac3d-464b-8542-e1cc8000610c\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " Jul 07 00:24:22.189840 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.189716 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-istio-data\") pod \"68cc7c09-ac3d-464b-8542-e1cc8000610c\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " Jul 07 00:24:22.189840 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.189748 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j69rb\" (UniqueName: \"kubernetes.io/projected/68cc7c09-ac3d-464b-8542-e1cc8000610c-kube-api-access-j69rb\") pod \"68cc7c09-ac3d-464b-8542-e1cc8000610c\" (UID: \"68cc7c09-ac3d-464b-8542-e1cc8000610c\") " Jul 07 00:24:22.190039 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.189843 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68cc7c09-ac3d-464b-8542-e1cc8000610c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "68cc7c09-ac3d-464b-8542-e1cc8000610c" (UID: "68cc7c09-ac3d-464b-8542-e1cc8000610c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:24:22.190039 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.189859 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "68cc7c09-ac3d-464b-8542-e1cc8000610c" (UID: "68cc7c09-ac3d-464b-8542-e1cc8000610c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:24:22.190039 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.189844 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "68cc7c09-ac3d-464b-8542-e1cc8000610c" (UID: "68cc7c09-ac3d-464b-8542-e1cc8000610c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:24:22.190039 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.190014 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "68cc7c09-ac3d-464b-8542-e1cc8000610c" (UID: "68cc7c09-ac3d-464b-8542-e1cc8000610c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:24:22.190329 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.190282 2553 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68cc7c09-ac3d-464b-8542-e1cc8000610c-istiod-ca-cert\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:24:22.190427 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.190334 2553 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-workload-certs\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:24:22.190427 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.190348 2553 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-workload-socket\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:24:22.190427 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.190360 2553 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-credential-socket\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:24:22.191645 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.191612 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-istio-data" (OuterVolumeSpecName: "istio-data") pod "68cc7c09-ac3d-464b-8542-e1cc8000610c" (UID: "68cc7c09-ac3d-464b-8542-e1cc8000610c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:24:22.192081 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.192046 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/68cc7c09-ac3d-464b-8542-e1cc8000610c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "68cc7c09-ac3d-464b-8542-e1cc8000610c" (UID: "68cc7c09-ac3d-464b-8542-e1cc8000610c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 00:24:22.192173 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.192064 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "68cc7c09-ac3d-464b-8542-e1cc8000610c" (UID: "68cc7c09-ac3d-464b-8542-e1cc8000610c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:24:22.192412 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.192395 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68cc7c09-ac3d-464b-8542-e1cc8000610c-kube-api-access-j69rb" (OuterVolumeSpecName: "kube-api-access-j69rb") pod "68cc7c09-ac3d-464b-8542-e1cc8000610c" (UID: "68cc7c09-ac3d-464b-8542-e1cc8000610c"). InnerVolumeSpecName "kube-api-access-j69rb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:24:22.192464 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.192419 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68cc7c09-ac3d-464b-8542-e1cc8000610c-istio-token" (OuterVolumeSpecName: "istio-token") pod "68cc7c09-ac3d-464b-8542-e1cc8000610c" (UID: "68cc7c09-ac3d-464b-8542-e1cc8000610c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:24:22.292082 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.292035 2553 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-istio-envoy\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:24:22.292082 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.292080 2553 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68cc7c09-ac3d-464b-8542-e1cc8000610c-istio-podinfo\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:24:22.292082 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.292092 2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68cc7c09-ac3d-464b-8542-e1cc8000610c-istio-token\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:24:22.292417 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.292103 2553 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68cc7c09-ac3d-464b-8542-e1cc8000610c-istio-data\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:24:22.292417 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.292118 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j69rb\" (UniqueName: \"kubernetes.io/projected/68cc7c09-ac3d-464b-8542-e1cc8000610c-kube-api-access-j69rb\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:24:22.735040 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.735003 2553 generic.go:356] "Generic (PLEG): container finished" podID="68cc7c09-ac3d-464b-8542-e1cc8000610c" containerID="b0922abffa0061b8a312a11336102a3865e6cf158c5c5b47bd63907d234b4830" exitCode=0 Jul 07 00:24:22.735231 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.735051 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" Jul 07 00:24:22.735231 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.735088 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" event={"ID":"68cc7c09-ac3d-464b-8542-e1cc8000610c","Type":"ContainerDied","Data":"b0922abffa0061b8a312a11336102a3865e6cf158c5c5b47bd63907d234b4830"} Jul 07 00:24:22.735231 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.735147 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh" event={"ID":"68cc7c09-ac3d-464b-8542-e1cc8000610c","Type":"ContainerDied","Data":"0866b2ad567b042da19e2e92cdd5e294f27a178f2ce86d48cdab9c48ee0b55b8"} Jul 07 00:24:22.735231 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.735172 2553 scope.go:117] "RemoveContainer" containerID="b0922abffa0061b8a312a11336102a3865e6cf158c5c5b47bd63907d234b4830" Jul 07 00:24:22.745399 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.745379 2553 scope.go:117] "RemoveContainer" containerID="0ce2b46fdfc3a1a403628e77862ccdc3b2b1960eb2dbfc9d64c5a3b92d98d7c5" Jul 07 00:24:22.755541 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.755519 2553 scope.go:117] "RemoveContainer" containerID="b0922abffa0061b8a312a11336102a3865e6cf158c5c5b47bd63907d234b4830" Jul 07 00:24:22.755846 ip-10-0-136-25 kubenswrapper[2553]: E0707 00:24:22.755826 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0922abffa0061b8a312a11336102a3865e6cf158c5c5b47bd63907d234b4830\": container with ID starting with b0922abffa0061b8a312a11336102a3865e6cf158c5c5b47bd63907d234b4830 not found: ID does not exist" containerID="b0922abffa0061b8a312a11336102a3865e6cf158c5c5b47bd63907d234b4830" Jul 07 00:24:22.755922 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.755857 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0922abffa0061b8a312a11336102a3865e6cf158c5c5b47bd63907d234b4830"} err="failed to get container status \"b0922abffa0061b8a312a11336102a3865e6cf158c5c5b47bd63907d234b4830\": rpc error: code = NotFound desc = could not find container \"b0922abffa0061b8a312a11336102a3865e6cf158c5c5b47bd63907d234b4830\": container with ID starting with b0922abffa0061b8a312a11336102a3865e6cf158c5c5b47bd63907d234b4830 not found: ID does not exist" Jul 07 00:24:22.755922 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.755878 2553 scope.go:117] "RemoveContainer" containerID="0ce2b46fdfc3a1a403628e77862ccdc3b2b1960eb2dbfc9d64c5a3b92d98d7c5" Jul 07 00:24:22.756695 ip-10-0-136-25 kubenswrapper[2553]: E0707 00:24:22.756278 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ce2b46fdfc3a1a403628e77862ccdc3b2b1960eb2dbfc9d64c5a3b92d98d7c5\": container with ID starting with 0ce2b46fdfc3a1a403628e77862ccdc3b2b1960eb2dbfc9d64c5a3b92d98d7c5 not found: ID does not exist" containerID="0ce2b46fdfc3a1a403628e77862ccdc3b2b1960eb2dbfc9d64c5a3b92d98d7c5" Jul 07 00:24:22.756822 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.756711 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ce2b46fdfc3a1a403628e77862ccdc3b2b1960eb2dbfc9d64c5a3b92d98d7c5"} err="failed to get container status \"0ce2b46fdfc3a1a403628e77862ccdc3b2b1960eb2dbfc9d64c5a3b92d98d7c5\": rpc error: code = NotFound desc = could not find container \"0ce2b46fdfc3a1a403628e77862ccdc3b2b1960eb2dbfc9d64c5a3b92d98d7c5\": container with ID starting with 0ce2b46fdfc3a1a403628e77862ccdc3b2b1960eb2dbfc9d64c5a3b92d98d7c5 not found: ID does not exist" Jul 07 00:24:22.757545 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.757523 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh"] Jul 07 00:24:22.760426 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:22.760406 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-28187d-openshift-default-6b49b767df-92kbh"] Jul 07 00:24:24.303426 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:24.303386 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68cc7c09-ac3d-464b-8542-e1cc8000610c" path="/var/lib/kubelet/pods/68cc7c09-ac3d-464b-8542-e1cc8000610c/volumes" Jul 07 00:24:25.890248 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:25.890211 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl"] Jul 07 00:24:25.890693 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:25.890677 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d690dd3b-e727-45e1-8053-0fa81d67574f" containerName="istio-proxy" Jul 07 00:24:25.890693 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:25.890690 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="d690dd3b-e727-45e1-8053-0fa81d67574f" containerName="istio-proxy" Jul 07 00:24:25.890764 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:25.890702 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68cc7c09-ac3d-464b-8542-e1cc8000610c" containerName="istio-proxy" Jul 07 00:24:25.890764 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:25.890707 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="68cc7c09-ac3d-464b-8542-e1cc8000610c" containerName="istio-proxy" Jul 07 00:24:25.890764 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:25.890723 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68cc7c09-ac3d-464b-8542-e1cc8000610c" containerName="istio-proxy" Jul 07 00:24:25.890764 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:25.890728 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="68cc7c09-ac3d-464b-8542-e1cc8000610c" containerName="istio-proxy" Jul 07 00:24:25.890891 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:25.890803 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="68cc7c09-ac3d-464b-8542-e1cc8000610c" containerName="istio-proxy" Jul 07 00:24:25.890891 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:25.890811 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="d690dd3b-e727-45e1-8053-0fa81d67574f" containerName="istio-proxy" Jul 07 00:24:25.894344 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:25.894318 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:25.897070 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:25.897048 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-accfb9-openshift-default-dockercfg-glpg7\"" Jul 07 00:24:25.903675 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:25.903631 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl"] Jul 07 00:24:26.032259 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.032141 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27e8723c-41bc-4cb7-a75a-94e50a58c677-istio-podinfo\") pod \"e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:26.032259 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.032196 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27e8723c-41bc-4cb7-a75a-94e50a58c677-istio-token\") pod \"e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:26.032259 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.032219 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxjmt\" (UniqueName: \"kubernetes.io/projected/27e8723c-41bc-4cb7-a75a-94e50a58c677-kube-api-access-nxjmt\") pod \"e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:26.032589 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.032333 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-istio-data\") pod \"e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:26.032589 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.032446 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-workload-certs\") pod \"e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:26.032589 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.032481 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27e8723c-41bc-4cb7-a75a-94e50a58c677-istiod-ca-cert\") pod \"e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:26.032589 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.032504 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-credential-socket\") pod \"e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:26.032747 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.032597 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-workload-socket\") pod \"e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:26.032747 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.032625 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-istio-envoy\") pod \"e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:26.133798 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.133754 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27e8723c-41bc-4cb7-a75a-94e50a58c677-istio-podinfo\") pod \"e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:26.134006 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.133809 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27e8723c-41bc-4cb7-a75a-94e50a58c677-istio-token\") pod \"e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:26.134006 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.133839 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxjmt\" (UniqueName: \"kubernetes.io/projected/27e8723c-41bc-4cb7-a75a-94e50a58c677-kube-api-access-nxjmt\") pod \"e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:26.134006 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.133873 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-istio-data\") pod \"e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:26.134006 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.133949 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-workload-certs\") pod \"e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:26.134006 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.133979 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27e8723c-41bc-4cb7-a75a-94e50a58c677-istiod-ca-cert\") pod \"e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:26.134006 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.134000 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-credential-socket\") pod \"e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:26.134349 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.134082 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-workload-socket\") pod \"e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:26.134349 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.134124 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-istio-envoy\") pod \"e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:26.134478 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.134449 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-workload-certs\") pod \"e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:26.134478 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.134467 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-istio-data\") pod \"e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:26.134592 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.134520 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-credential-socket\") pod \"e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:26.134667 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.134644 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-workload-socket\") pod \"e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:26.134812 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.134792 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27e8723c-41bc-4cb7-a75a-94e50a58c677-istiod-ca-cert\") pod \"e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:26.136487 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.136455 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-istio-envoy\") pod \"e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:26.136633 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.136499 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27e8723c-41bc-4cb7-a75a-94e50a58c677-istio-podinfo\") pod \"e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:26.142582 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.142551 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27e8723c-41bc-4cb7-a75a-94e50a58c677-istio-token\") pod \"e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:26.142740 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.142665 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxjmt\" (UniqueName: \"kubernetes.io/projected/27e8723c-41bc-4cb7-a75a-94e50a58c677-kube-api-access-nxjmt\") pod \"e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:26.206774 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.206742 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:26.365066 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.365038 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl"] Jul 07 00:24:26.366486 ip-10-0-136-25 kubenswrapper[2553]: W0707 00:24:26.366458 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27e8723c_41bc_4cb7_a75a_94e50a58c677.slice/crio-d0bac54a124e94aa3b02929bb977c75451bb79b4162be38788e0a7a24296ab38 WatchSource:0}: Error finding container d0bac54a124e94aa3b02929bb977c75451bb79b4162be38788e0a7a24296ab38: Status 404 returned error can't find the container with id d0bac54a124e94aa3b02929bb977c75451bb79b4162be38788e0a7a24296ab38 Jul 07 00:24:26.369631 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.369598 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:24:26.369723 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.369670 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:24:26.369723 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.369698 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:24:26.762323 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.762255 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" event={"ID":"27e8723c-41bc-4cb7-a75a-94e50a58c677","Type":"ContainerStarted","Data":"e6ca4de18f30c4792975e5fe3c7430a074df52956a3e0a9ac7dabe13ab35ad71"} Jul 07 00:24:26.762519 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.762334 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" event={"ID":"27e8723c-41bc-4cb7-a75a-94e50a58c677","Type":"ContainerStarted","Data":"d0bac54a124e94aa3b02929bb977c75451bb79b4162be38788e0a7a24296ab38"} Jul 07 00:24:26.788636 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:26.788568 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" podStartSLOduration=1.7885449260000001 podStartE2EDuration="1.788544926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:24:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:24:26.784241206 +0000 UTC m=+2148.993970721" watchObservedRunningTime="2026-07-07 00:24:26.788544926 +0000 UTC m=+2148.998274450" Jul 07 00:24:27.207945 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:27.207909 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:27.213171 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:27.213144 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:27.769179 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:27.769143 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:27.770263 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:27.770239 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:34.951637 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:34.951001 2553 patch_prober.go:28] interesting pod/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": read tcp 10.132.0.2:39778->10.132.0.75:15021: read: connection reset by peer" start-of-body= Jul 07 00:24:34.951637 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:34.951064 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" podUID="4787a56a-3a04-45b1-91a9-c44a9883025d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": read tcp 10.132.0.2:39778->10.132.0.75:15021: read: connection reset by peer" Jul 07 00:24:35.827001 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:35.826957 2553 generic.go:356] "Generic (PLEG): container finished" podID="4787a56a-3a04-45b1-91a9-c44a9883025d" containerID="ce585010bb3521df6906c87747043b03f97c3011ad5855bbc4659a16053cb203" exitCode=0 Jul 07 00:24:35.827223 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:35.827038 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" event={"ID":"4787a56a-3a04-45b1-91a9-c44a9883025d","Type":"ContainerDied","Data":"ce585010bb3521df6906c87747043b03f97c3011ad5855bbc4659a16053cb203"} Jul 07 00:24:35.828002 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:35.827977 2553 scope.go:117] "RemoveContainer" containerID="ce585010bb3521df6906c87747043b03f97c3011ad5855bbc4659a16053cb203" Jul 07 00:24:35.829879 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:35.829842 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:24:35.830124 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:35.830098 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:24:35.830284 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:35.830264 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:24:36.839520 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:36.839480 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" event={"ID":"4787a56a-3a04-45b1-91a9-c44a9883025d","Type":"ContainerStarted","Data":"4dca3f6dd073fdeca3c2390ccb36cdc0419aa7ac56241a6c91c74ee1cbc6c4d9"} Jul 07 00:24:37.167841 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:37.167803 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:37.168327 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:37.168239 2553 patch_prober.go:28] interesting pod/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Jul 07 00:24:37.168519 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:37.168492 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" podUID="4787a56a-3a04-45b1-91a9-c44a9883025d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Jul 07 00:24:38.168345 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:38.168269 2553 patch_prober.go:28] interesting pod/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Jul 07 00:24:38.168881 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:38.168378 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" podUID="4787a56a-3a04-45b1-91a9-c44a9883025d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Jul 07 00:24:39.167633 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:39.167596 2553 patch_prober.go:28] interesting pod/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Jul 07 00:24:39.167824 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:39.167668 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" podUID="4787a56a-3a04-45b1-91a9-c44a9883025d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Jul 07 00:24:40.168335 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:40.168274 2553 patch_prober.go:28] interesting pod/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Jul 07 00:24:40.168733 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:40.168355 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" podUID="4787a56a-3a04-45b1-91a9-c44a9883025d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Jul 07 00:24:41.171735 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:41.171700 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:41.172174 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:41.171996 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:41.172900 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:41.172878 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:24:42.770346 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:42.770273 2553 patch_prober.go:28] interesting pod/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Jul 07 00:24:42.770870 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:42.770367 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" podUID="27e8723c-41bc-4cb7-a75a-94e50a58c677" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Jul 07 00:24:42.900856 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:42.900818 2553 generic.go:356] "Generic (PLEG): container finished" podID="27e8723c-41bc-4cb7-a75a-94e50a58c677" containerID="e6ca4de18f30c4792975e5fe3c7430a074df52956a3e0a9ac7dabe13ab35ad71" exitCode=0 Jul 07 00:24:42.901040 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:42.900884 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" event={"ID":"27e8723c-41bc-4cb7-a75a-94e50a58c677","Type":"ContainerDied","Data":"e6ca4de18f30c4792975e5fe3c7430a074df52956a3e0a9ac7dabe13ab35ad71"} Jul 07 00:24:42.901457 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:42.901442 2553 scope.go:117] "RemoveContainer" containerID="e6ca4de18f30c4792975e5fe3c7430a074df52956a3e0a9ac7dabe13ab35ad71" Jul 07 00:24:42.902469 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:42.902443 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:24:42.902528 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:42.902513 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:24:42.902570 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:42.902545 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 00:24:43.908680 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:43.908640 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" event={"ID":"27e8723c-41bc-4cb7-a75a-94e50a58c677","Type":"ContainerStarted","Data":"9d7cbeda20fda82635f503c11cab080d1156ff7f0a668e6e1854cd458c6ba6c6"} Jul 07 00:24:44.207645 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:44.207555 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:44.207818 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:44.207748 2553 patch_prober.go:28] interesting pod/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Jul 07 00:24:44.207818 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:44.207795 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" podUID="27e8723c-41bc-4cb7-a75a-94e50a58c677" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Jul 07 00:24:45.207645 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:45.207603 2553 patch_prober.go:28] interesting pod/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Jul 07 00:24:45.208093 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:45.207687 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" podUID="27e8723c-41bc-4cb7-a75a-94e50a58c677" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Jul 07 00:24:46.207342 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:46.207281 2553 patch_prober.go:28] interesting pod/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Jul 07 00:24:46.207525 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:46.207362 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" podUID="27e8723c-41bc-4cb7-a75a-94e50a58c677" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Jul 07 00:24:47.207845 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:47.207807 2553 patch_prober.go:28] interesting pod/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Jul 07 00:24:47.208263 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:47.207866 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" podUID="27e8723c-41bc-4cb7-a75a-94e50a58c677" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Jul 07 00:24:48.211672 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:48.211640 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:48.212145 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:48.211925 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:24:48.212672 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:24:48.212654 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:28:38.468645 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:28:38.468617 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4z8vq_3253bdfe-5248-4d4c-9864-11660eb042ca/ovn-acl-logging/0.log" Jul 07 00:28:38.473638 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:28:38.473615 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4z8vq_3253bdfe-5248-4d4c-9864-11660eb042ca/ovn-acl-logging/0.log" Jul 07 00:30:35.337252 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:35.337147 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77"] Jul 07 00:30:35.337815 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:35.337487 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" podUID="4787a56a-3a04-45b1-91a9-c44a9883025d" containerName="istio-proxy" containerID="cri-o://4dca3f6dd073fdeca3c2390ccb36cdc0419aa7ac56241a6c91c74ee1cbc6c4d9" gracePeriod=30 Jul 07 00:30:40.609793 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:40.609719 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:30:40.670372 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:40.670342 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-workload-certs\") pod \"4787a56a-3a04-45b1-91a9-c44a9883025d\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " Jul 07 00:30:40.670372 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:40.670378 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-workload-socket\") pod \"4787a56a-3a04-45b1-91a9-c44a9883025d\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " Jul 07 00:30:40.670648 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:40.670400 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvv9z\" (UniqueName: \"kubernetes.io/projected/4787a56a-3a04-45b1-91a9-c44a9883025d-kube-api-access-jvv9z\") pod \"4787a56a-3a04-45b1-91a9-c44a9883025d\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " Jul 07 00:30:40.670648 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:40.670433 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4787a56a-3a04-45b1-91a9-c44a9883025d-istiod-ca-cert\") pod \"4787a56a-3a04-45b1-91a9-c44a9883025d\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " Jul 07 00:30:40.670648 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:40.670453 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4787a56a-3a04-45b1-91a9-c44a9883025d-istio-token\") pod \"4787a56a-3a04-45b1-91a9-c44a9883025d\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " Jul 07 00:30:40.670648 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:40.670475 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-credential-socket\") pod \"4787a56a-3a04-45b1-91a9-c44a9883025d\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " Jul 07 00:30:40.670648 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:40.670500 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4787a56a-3a04-45b1-91a9-c44a9883025d-istio-podinfo\") pod \"4787a56a-3a04-45b1-91a9-c44a9883025d\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " Jul 07 00:30:40.670648 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:40.670543 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-istio-envoy\") pod \"4787a56a-3a04-45b1-91a9-c44a9883025d\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " Jul 07 00:30:40.670648 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:40.670618 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-istio-data\") pod \"4787a56a-3a04-45b1-91a9-c44a9883025d\" (UID: \"4787a56a-3a04-45b1-91a9-c44a9883025d\") " Jul 07 00:30:40.670963 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:40.670718 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4787a56a-3a04-45b1-91a9-c44a9883025d" (UID: "4787a56a-3a04-45b1-91a9-c44a9883025d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:30:40.670963 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:40.670755 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4787a56a-3a04-45b1-91a9-c44a9883025d" (UID: "4787a56a-3a04-45b1-91a9-c44a9883025d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:30:40.671054 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:40.671038 2553 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-workload-certs\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:30:40.671111 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:40.671059 2553 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-workload-socket\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:30:40.671171 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:40.671142 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4787a56a-3a04-45b1-91a9-c44a9883025d" (UID: "4787a56a-3a04-45b1-91a9-c44a9883025d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:30:40.671228 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:40.671189 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4787a56a-3a04-45b1-91a9-c44a9883025d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4787a56a-3a04-45b1-91a9-c44a9883025d" (UID: "4787a56a-3a04-45b1-91a9-c44a9883025d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:30:40.671614 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:40.671587 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-istio-data" (OuterVolumeSpecName: "istio-data") pod "4787a56a-3a04-45b1-91a9-c44a9883025d" (UID: "4787a56a-3a04-45b1-91a9-c44a9883025d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:30:40.672955 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:40.672936 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4787a56a-3a04-45b1-91a9-c44a9883025d" (UID: "4787a56a-3a04-45b1-91a9-c44a9883025d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:30:40.673100 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:40.672997 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4787a56a-3a04-45b1-91a9-c44a9883025d-kube-api-access-jvv9z" (OuterVolumeSpecName: "kube-api-access-jvv9z") pod "4787a56a-3a04-45b1-91a9-c44a9883025d" (UID: "4787a56a-3a04-45b1-91a9-c44a9883025d"). InnerVolumeSpecName "kube-api-access-jvv9z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:30:40.673392 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:40.673372 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4787a56a-3a04-45b1-91a9-c44a9883025d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4787a56a-3a04-45b1-91a9-c44a9883025d" (UID: "4787a56a-3a04-45b1-91a9-c44a9883025d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 00:30:40.673451 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:40.673390 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4787a56a-3a04-45b1-91a9-c44a9883025d-istio-token" (OuterVolumeSpecName: "istio-token") pod "4787a56a-3a04-45b1-91a9-c44a9883025d" (UID: "4787a56a-3a04-45b1-91a9-c44a9883025d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:30:40.772015 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:40.771977 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvv9z\" (UniqueName: \"kubernetes.io/projected/4787a56a-3a04-45b1-91a9-c44a9883025d-kube-api-access-jvv9z\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:30:40.772015 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:40.772008 2553 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4787a56a-3a04-45b1-91a9-c44a9883025d-istiod-ca-cert\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:30:40.772015 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:40.772017 2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4787a56a-3a04-45b1-91a9-c44a9883025d-istio-token\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:30:40.772271 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:40.772027 2553 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-credential-socket\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:30:40.772271 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:40.772035 2553 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4787a56a-3a04-45b1-91a9-c44a9883025d-istio-podinfo\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:30:40.772271 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:40.772043 2553 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-istio-envoy\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:30:40.772271 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:40.772050 2553 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4787a56a-3a04-45b1-91a9-c44a9883025d-istio-data\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:30:41.087455 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:41.087410 2553 generic.go:356] "Generic (PLEG): container finished" podID="4787a56a-3a04-45b1-91a9-c44a9883025d" containerID="4dca3f6dd073fdeca3c2390ccb36cdc0419aa7ac56241a6c91c74ee1cbc6c4d9" exitCode=0 Jul 07 00:30:41.087645 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:41.087465 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" Jul 07 00:30:41.087645 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:41.087499 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" event={"ID":"4787a56a-3a04-45b1-91a9-c44a9883025d","Type":"ContainerDied","Data":"4dca3f6dd073fdeca3c2390ccb36cdc0419aa7ac56241a6c91c74ee1cbc6c4d9"} Jul 07 00:30:41.087645 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:41.087565 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77" event={"ID":"4787a56a-3a04-45b1-91a9-c44a9883025d","Type":"ContainerDied","Data":"d8c4963b7e23848aa21f9930723e14a9a5098cf0c8e75420e592f752cad46ea9"} Jul 07 00:30:41.087645 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:41.087595 2553 scope.go:117] "RemoveContainer" containerID="4dca3f6dd073fdeca3c2390ccb36cdc0419aa7ac56241a6c91c74ee1cbc6c4d9" Jul 07 00:30:41.098766 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:41.098748 2553 scope.go:117] "RemoveContainer" containerID="ce585010bb3521df6906c87747043b03f97c3011ad5855bbc4659a16053cb203" Jul 07 00:30:41.108173 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:41.108154 2553 scope.go:117] "RemoveContainer" containerID="4dca3f6dd073fdeca3c2390ccb36cdc0419aa7ac56241a6c91c74ee1cbc6c4d9" Jul 07 00:30:41.108528 ip-10-0-136-25 kubenswrapper[2553]: E0707 00:30:41.108500 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dca3f6dd073fdeca3c2390ccb36cdc0419aa7ac56241a6c91c74ee1cbc6c4d9\": container with ID starting with 4dca3f6dd073fdeca3c2390ccb36cdc0419aa7ac56241a6c91c74ee1cbc6c4d9 not found: ID does not exist" containerID="4dca3f6dd073fdeca3c2390ccb36cdc0419aa7ac56241a6c91c74ee1cbc6c4d9" Jul 07 00:30:41.108663 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:41.108538 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dca3f6dd073fdeca3c2390ccb36cdc0419aa7ac56241a6c91c74ee1cbc6c4d9"} err="failed to get container status \"4dca3f6dd073fdeca3c2390ccb36cdc0419aa7ac56241a6c91c74ee1cbc6c4d9\": rpc error: code = NotFound desc = could not find container \"4dca3f6dd073fdeca3c2390ccb36cdc0419aa7ac56241a6c91c74ee1cbc6c4d9\": container with ID starting with 4dca3f6dd073fdeca3c2390ccb36cdc0419aa7ac56241a6c91c74ee1cbc6c4d9 not found: ID does not exist" Jul 07 00:30:41.108663 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:41.108561 2553 scope.go:117] "RemoveContainer" containerID="ce585010bb3521df6906c87747043b03f97c3011ad5855bbc4659a16053cb203" Jul 07 00:30:41.108971 ip-10-0-136-25 kubenswrapper[2553]: E0707 00:30:41.108904 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce585010bb3521df6906c87747043b03f97c3011ad5855bbc4659a16053cb203\": container with ID starting with ce585010bb3521df6906c87747043b03f97c3011ad5855bbc4659a16053cb203 not found: ID does not exist" containerID="ce585010bb3521df6906c87747043b03f97c3011ad5855bbc4659a16053cb203" Jul 07 00:30:41.108971 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:41.108944 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce585010bb3521df6906c87747043b03f97c3011ad5855bbc4659a16053cb203"} err="failed to get container status \"ce585010bb3521df6906c87747043b03f97c3011ad5855bbc4659a16053cb203\": rpc error: code = NotFound desc = could not find container \"ce585010bb3521df6906c87747043b03f97c3011ad5855bbc4659a16053cb203\": container with ID starting with ce585010bb3521df6906c87747043b03f97c3011ad5855bbc4659a16053cb203 not found: ID does not exist" Jul 07 00:30:41.112264 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:41.112242 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77"] Jul 07 00:30:41.118326 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:41.118289 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-fced7e-openshift-default-569d87dd75-4ml77"] Jul 07 00:30:42.258115 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:42.258080 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl"] Jul 07 00:30:42.258639 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:42.258375 2553 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" podUID="27e8723c-41bc-4cb7-a75a-94e50a58c677" containerName="istio-proxy" containerID="cri-o://9d7cbeda20fda82635f503c11cab080d1156ff7f0a668e6e1854cd458c6ba6c6" gracePeriod=30 Jul 07 00:30:42.303484 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:42.303447 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4787a56a-3a04-45b1-91a9-c44a9883025d" path="/var/lib/kubelet/pods/4787a56a-3a04-45b1-91a9-c44a9883025d/volumes" Jul 07 00:30:47.524289 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:47.524222 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:30:47.645765 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:47.645722 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-workload-certs\") pod \"27e8723c-41bc-4cb7-a75a-94e50a58c677\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " Jul 07 00:30:47.645765 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:47.645775 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27e8723c-41bc-4cb7-a75a-94e50a58c677-istiod-ca-cert\") pod \"27e8723c-41bc-4cb7-a75a-94e50a58c677\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " Jul 07 00:30:47.646035 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:47.645845 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nxjmt\" (UniqueName: \"kubernetes.io/projected/27e8723c-41bc-4cb7-a75a-94e50a58c677-kube-api-access-nxjmt\") pod \"27e8723c-41bc-4cb7-a75a-94e50a58c677\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " Jul 07 00:30:47.646035 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:47.645908 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-istio-envoy\") pod \"27e8723c-41bc-4cb7-a75a-94e50a58c677\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " Jul 07 00:30:47.646035 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:47.645941 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-credential-socket\") pod \"27e8723c-41bc-4cb7-a75a-94e50a58c677\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " Jul 07 00:30:47.646035 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:47.645974 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27e8723c-41bc-4cb7-a75a-94e50a58c677-istio-podinfo\") pod \"27e8723c-41bc-4cb7-a75a-94e50a58c677\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " Jul 07 00:30:47.646035 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:47.646008 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-workload-socket\") pod \"27e8723c-41bc-4cb7-a75a-94e50a58c677\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " Jul 07 00:30:47.646334 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:47.646043 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27e8723c-41bc-4cb7-a75a-94e50a58c677-istio-token\") pod \"27e8723c-41bc-4cb7-a75a-94e50a58c677\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " Jul 07 00:30:47.646334 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:47.646071 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-istio-data\") pod \"27e8723c-41bc-4cb7-a75a-94e50a58c677\" (UID: \"27e8723c-41bc-4cb7-a75a-94e50a58c677\") " Jul 07 00:30:47.646334 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:47.646069 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "27e8723c-41bc-4cb7-a75a-94e50a58c677" (UID: "27e8723c-41bc-4cb7-a75a-94e50a58c677"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:30:47.646516 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:47.646344 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27e8723c-41bc-4cb7-a75a-94e50a58c677-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "27e8723c-41bc-4cb7-a75a-94e50a58c677" (UID: "27e8723c-41bc-4cb7-a75a-94e50a58c677"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:30:47.646571 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:47.646530 2553 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-workload-certs\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:30:47.646571 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:47.646554 2553 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27e8723c-41bc-4cb7-a75a-94e50a58c677-istiod-ca-cert\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:30:47.646571 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:47.646532 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "27e8723c-41bc-4cb7-a75a-94e50a58c677" (UID: "27e8723c-41bc-4cb7-a75a-94e50a58c677"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:30:47.646739 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:47.646466 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "27e8723c-41bc-4cb7-a75a-94e50a58c677" (UID: "27e8723c-41bc-4cb7-a75a-94e50a58c677"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:30:47.646984 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:47.646963 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-istio-data" (OuterVolumeSpecName: "istio-data") pod "27e8723c-41bc-4cb7-a75a-94e50a58c677" (UID: "27e8723c-41bc-4cb7-a75a-94e50a58c677"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:30:47.648279 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:47.648254 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27e8723c-41bc-4cb7-a75a-94e50a58c677-istio-token" (OuterVolumeSpecName: "istio-token") pod "27e8723c-41bc-4cb7-a75a-94e50a58c677" (UID: "27e8723c-41bc-4cb7-a75a-94e50a58c677"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:30:47.648456 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:47.648286 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "27e8723c-41bc-4cb7-a75a-94e50a58c677" (UID: "27e8723c-41bc-4cb7-a75a-94e50a58c677"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:30:47.648653 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:47.648634 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/27e8723c-41bc-4cb7-a75a-94e50a58c677-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "27e8723c-41bc-4cb7-a75a-94e50a58c677" (UID: "27e8723c-41bc-4cb7-a75a-94e50a58c677"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 00:30:47.648700 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:47.648673 2553 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27e8723c-41bc-4cb7-a75a-94e50a58c677-kube-api-access-nxjmt" (OuterVolumeSpecName: "kube-api-access-nxjmt") pod "27e8723c-41bc-4cb7-a75a-94e50a58c677" (UID: "27e8723c-41bc-4cb7-a75a-94e50a58c677"). InnerVolumeSpecName "kube-api-access-nxjmt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:30:47.747408 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:47.747370 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxjmt\" (UniqueName: \"kubernetes.io/projected/27e8723c-41bc-4cb7-a75a-94e50a58c677-kube-api-access-nxjmt\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:30:47.747408 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:47.747401 2553 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-istio-envoy\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:30:47.747408 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:47.747412 2553 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-credential-socket\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:30:47.747668 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:47.747422 2553 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27e8723c-41bc-4cb7-a75a-94e50a58c677-istio-podinfo\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:30:47.747668 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:47.747431 2553 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-workload-socket\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:30:47.747668 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:47.747440 2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27e8723c-41bc-4cb7-a75a-94e50a58c677-istio-token\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:30:47.747668 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:47.747449 2553 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27e8723c-41bc-4cb7-a75a-94e50a58c677-istio-data\") on node \"ip-10-0-136-25.ec2.internal\" DevicePath \"\"" Jul 07 00:30:48.048072 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:48.048032 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-szrs8_6df592ff-ca58-4349-9af2-0d2175c2e995/manager/0.log" Jul 07 00:30:48.134112 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:48.134075 2553 generic.go:356] "Generic (PLEG): container finished" podID="27e8723c-41bc-4cb7-a75a-94e50a58c677" containerID="9d7cbeda20fda82635f503c11cab080d1156ff7f0a668e6e1854cd458c6ba6c6" exitCode=0 Jul 07 00:30:48.134341 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:48.134122 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" Jul 07 00:30:48.134341 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:48.134123 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" event={"ID":"27e8723c-41bc-4cb7-a75a-94e50a58c677","Type":"ContainerDied","Data":"9d7cbeda20fda82635f503c11cab080d1156ff7f0a668e6e1854cd458c6ba6c6"} Jul 07 00:30:48.134341 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:48.134235 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl" event={"ID":"27e8723c-41bc-4cb7-a75a-94e50a58c677","Type":"ContainerDied","Data":"d0bac54a124e94aa3b02929bb977c75451bb79b4162be38788e0a7a24296ab38"} Jul 07 00:30:48.134341 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:48.134267 2553 scope.go:117] "RemoveContainer" containerID="9d7cbeda20fda82635f503c11cab080d1156ff7f0a668e6e1854cd458c6ba6c6" Jul 07 00:30:48.144552 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:48.144526 2553 scope.go:117] "RemoveContainer" containerID="e6ca4de18f30c4792975e5fe3c7430a074df52956a3e0a9ac7dabe13ab35ad71" Jul 07 00:30:48.153500 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:48.153361 2553 scope.go:117] "RemoveContainer" containerID="9d7cbeda20fda82635f503c11cab080d1156ff7f0a668e6e1854cd458c6ba6c6" Jul 07 00:30:48.153679 ip-10-0-136-25 kubenswrapper[2553]: E0707 00:30:48.153652 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d7cbeda20fda82635f503c11cab080d1156ff7f0a668e6e1854cd458c6ba6c6\": container with ID starting with 9d7cbeda20fda82635f503c11cab080d1156ff7f0a668e6e1854cd458c6ba6c6 not found: ID does not exist" containerID="9d7cbeda20fda82635f503c11cab080d1156ff7f0a668e6e1854cd458c6ba6c6" Jul 07 00:30:48.153787 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:48.153686 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d7cbeda20fda82635f503c11cab080d1156ff7f0a668e6e1854cd458c6ba6c6"} err="failed to get container status \"9d7cbeda20fda82635f503c11cab080d1156ff7f0a668e6e1854cd458c6ba6c6\": rpc error: code = NotFound desc = could not find container \"9d7cbeda20fda82635f503c11cab080d1156ff7f0a668e6e1854cd458c6ba6c6\": container with ID starting with 9d7cbeda20fda82635f503c11cab080d1156ff7f0a668e6e1854cd458c6ba6c6 not found: ID does not exist" Jul 07 00:30:48.153787 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:48.153704 2553 scope.go:117] "RemoveContainer" containerID="e6ca4de18f30c4792975e5fe3c7430a074df52956a3e0a9ac7dabe13ab35ad71" Jul 07 00:30:48.153951 ip-10-0-136-25 kubenswrapper[2553]: E0707 00:30:48.153933 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6ca4de18f30c4792975e5fe3c7430a074df52956a3e0a9ac7dabe13ab35ad71\": container with ID starting with e6ca4de18f30c4792975e5fe3c7430a074df52956a3e0a9ac7dabe13ab35ad71 not found: ID does not exist" containerID="e6ca4de18f30c4792975e5fe3c7430a074df52956a3e0a9ac7dabe13ab35ad71" Jul 07 00:30:48.153990 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:48.153957 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6ca4de18f30c4792975e5fe3c7430a074df52956a3e0a9ac7dabe13ab35ad71"} err="failed to get container status \"e6ca4de18f30c4792975e5fe3c7430a074df52956a3e0a9ac7dabe13ab35ad71\": rpc error: code = NotFound desc = could not find container \"e6ca4de18f30c4792975e5fe3c7430a074df52956a3e0a9ac7dabe13ab35ad71\": container with ID starting with e6ca4de18f30c4792975e5fe3c7430a074df52956a3e0a9ac7dabe13ab35ad71 not found: ID does not exist" Jul 07 00:30:48.158129 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:48.158105 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl"] Jul 07 00:30:48.162813 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:48.162784 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-accfb9-openshift-default-5f99476475-qnnbl"] Jul 07 00:30:48.177896 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:48.177868 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-bwhj6_e4805476-0013-484a-b046-999b6308296f/manager/0.log" Jul 07 00:30:48.302510 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:48.302424 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-7bdf95f944-bdxls_818a2117-421b-478b-9633-a95a0a927158/maas-api/0.log" Jul 07 00:30:48.304615 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:48.304584 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27e8723c-41bc-4cb7-a75a-94e50a58c677" path="/var/lib/kubelet/pods/27e8723c-41bc-4cb7-a75a-94e50a58c677/volumes" Jul 07 00:30:48.413124 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:48.413094 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723040-s84z8_34d89dc6-76e4-42c1-846a-6708ea0d87ea/cleanup/0.log" Jul 07 00:30:48.784919 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:48.784888 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-56bf7f957f-sjkb7_52bd584a-8a0e-462f-9ae5-79c5295eab77/manager/0.log" Jul 07 00:30:49.418553 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:49.418526 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bcdcfc745-xcv6b_0209cf45-9d4a-4f99-8cdb-29e120237003/copy-manifests/0.log" Jul 07 00:30:49.432772 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:49.432747 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bcdcfc745-xcv6b_0209cf45-9d4a-4f99-8cdb-29e120237003/manager/0.log" Jul 07 00:30:49.557241 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:49.557201 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-mjkm4_d27d869f-2a23-476d-9006-3e5c71823782/postgres/0.log" Jul 07 00:30:50.371113 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:50.371071 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c_d4edb7a7-2478-43e1-b346-d786e1606487/util/0.log" Jul 07 00:30:50.378529 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:50.378504 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c_d4edb7a7-2478-43e1-b346-d786e1606487/pull/0.log" Jul 07 00:30:50.385288 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:50.385266 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c_d4edb7a7-2478-43e1-b346-d786e1606487/extract/0.log" Jul 07 00:30:50.494980 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:50.494948 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm_c4be0703-a52e-49d4-8942-edbb447df0ad/util/0.log" Jul 07 00:30:50.501991 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:50.501972 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm_c4be0703-a52e-49d4-8942-edbb447df0ad/pull/0.log" Jul 07 00:30:50.509140 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:50.509121 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm_c4be0703-a52e-49d4-8942-edbb447df0ad/extract/0.log" Jul 07 00:30:50.620837 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:50.620799 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s_7b1c8289-6786-41c6-a0fd-66e613080735/util/0.log" Jul 07 00:30:50.628054 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:50.627974 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s_7b1c8289-6786-41c6-a0fd-66e613080735/pull/0.log" Jul 07 00:30:50.635076 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:50.635056 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s_7b1c8289-6786-41c6-a0fd-66e613080735/extract/0.log" Jul 07 00:30:50.752397 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:50.752362 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v_a3ced335-f964-46a9-b311-969a96fc1136/util/0.log" Jul 07 00:30:50.759167 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:50.759146 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v_a3ced335-f964-46a9-b311-969a96fc1136/pull/0.log" Jul 07 00:30:50.766455 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:50.766435 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v_a3ced335-f964-46a9-b311-969a96fc1136/extract/0.log" Jul 07 00:30:51.010339 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:51.010309 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-xlhnc_a58195c7-3ab7-4748-a16f-58f9f6d1d23e/manager/0.log" Jul 07 00:30:51.136754 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:51.136726 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-sw76w_e090dac8-d5d5-4ea9-a0c2-1ffec011d9e2/manager/0.log" Jul 07 00:30:51.382965 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:51.382880 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-pmjrz_8708739e-7b51-4394-9e97-89a8c1b87997/registry-server/0.log" Jul 07 00:30:51.516938 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:51.516909 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-7pmgv_bf17cf71-15c2-40b4-81bb-df10ca03011c/manager/0.log" Jul 07 00:30:51.793518 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:51.793488 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-d8fs6_ee864a87-f13d-418c-9f4e-43dc12ce6add/manager/0.log" Jul 07 00:30:52.178441 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:52.178415 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678sppw9_379fe0e0-dc84-4b69-a99d-dcf861e32ce6/istio-proxy/0.log" Jul 07 00:30:52.429952 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:52.429877 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dc8c455ff-sbm28_b94f94fd-f309-4495-86af-fb0eb3946b4f/kube-auth-proxy/0.log" Jul 07 00:30:52.683524 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:52.683455 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-7x84r_867fe4ad-91d7-4301-b82a-f5eaf31f863d/istio-proxy/1.log" Jul 07 00:30:52.814505 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:52.814472 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-tmjqm_c6582771-54aa-4c55-b416-3f661db37add/payload-pre-processing/0.log" Jul 07 00:30:52.937514 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:52.937430 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-2bhlm_e9cf329f-b667-4b93-8511-811c1f1efee3/payload-processing/0.log" Jul 07 00:30:53.684216 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:53.684192 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-jkbft_41218329-f2c7-4623-b636-c99ec3352819/main/0.log" Jul 07 00:30:53.915160 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:53.915136 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-4dhff_b5f3159e-717f-4263-8ce9-b317ecb5d8c4/main/0.log" Jul 07 00:30:58.803922 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:58.803884 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nrxzw/must-gather-5k4zj"] Jul 07 00:30:58.804515 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:58.804337 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27e8723c-41bc-4cb7-a75a-94e50a58c677" containerName="istio-proxy" Jul 07 00:30:58.804515 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:58.804351 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="27e8723c-41bc-4cb7-a75a-94e50a58c677" containerName="istio-proxy" Jul 07 00:30:58.804515 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:58.804361 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27e8723c-41bc-4cb7-a75a-94e50a58c677" containerName="istio-proxy" Jul 07 00:30:58.804515 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:58.804366 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="27e8723c-41bc-4cb7-a75a-94e50a58c677" containerName="istio-proxy" Jul 07 00:30:58.804515 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:58.804382 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4787a56a-3a04-45b1-91a9-c44a9883025d" containerName="istio-proxy" Jul 07 00:30:58.804515 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:58.804388 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="4787a56a-3a04-45b1-91a9-c44a9883025d" containerName="istio-proxy" Jul 07 00:30:58.804515 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:58.804397 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4787a56a-3a04-45b1-91a9-c44a9883025d" containerName="istio-proxy" Jul 07 00:30:58.804515 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:58.804402 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="4787a56a-3a04-45b1-91a9-c44a9883025d" containerName="istio-proxy" Jul 07 00:30:58.804515 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:58.804465 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="4787a56a-3a04-45b1-91a9-c44a9883025d" containerName="istio-proxy" Jul 07 00:30:58.804515 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:58.804472 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="4787a56a-3a04-45b1-91a9-c44a9883025d" containerName="istio-proxy" Jul 07 00:30:58.804515 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:58.804480 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="27e8723c-41bc-4cb7-a75a-94e50a58c677" containerName="istio-proxy" Jul 07 00:30:58.804515 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:58.804485 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="27e8723c-41bc-4cb7-a75a-94e50a58c677" containerName="istio-proxy" Jul 07 00:30:58.804515 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:58.804492 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="68cc7c09-ac3d-464b-8542-e1cc8000610c" containerName="istio-proxy" Jul 07 00:30:58.809285 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:58.809268 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nrxzw/must-gather-5k4zj" Jul 07 00:30:58.812510 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:58.812486 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nrxzw\"/\"openshift-service-ca.crt\"" Jul 07 00:30:58.813178 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:58.813161 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nrxzw\"/\"kube-root-ca.crt\"" Jul 07 00:30:58.813765 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:58.813749 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nrxzw\"/\"default-dockercfg-68rc2\"" Jul 07 00:30:58.820544 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:58.820518 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nrxzw/must-gather-5k4zj"] Jul 07 00:30:58.952654 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:58.952621 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b067ae86-e310-467b-a0cd-f36b3ce05360-must-gather-output\") pod \"must-gather-5k4zj\" (UID: \"b067ae86-e310-467b-a0cd-f36b3ce05360\") " pod="openshift-must-gather-nrxzw/must-gather-5k4zj" Jul 07 00:30:58.952654 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:58.952658 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n77lc\" (UniqueName: \"kubernetes.io/projected/b067ae86-e310-467b-a0cd-f36b3ce05360-kube-api-access-n77lc\") pod \"must-gather-5k4zj\" (UID: \"b067ae86-e310-467b-a0cd-f36b3ce05360\") " pod="openshift-must-gather-nrxzw/must-gather-5k4zj" Jul 07 00:30:59.053853 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:59.053807 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b067ae86-e310-467b-a0cd-f36b3ce05360-must-gather-output\") pod \"must-gather-5k4zj\" (UID: \"b067ae86-e310-467b-a0cd-f36b3ce05360\") " pod="openshift-must-gather-nrxzw/must-gather-5k4zj" Jul 07 00:30:59.053853 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:59.053851 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n77lc\" (UniqueName: \"kubernetes.io/projected/b067ae86-e310-467b-a0cd-f36b3ce05360-kube-api-access-n77lc\") pod \"must-gather-5k4zj\" (UID: \"b067ae86-e310-467b-a0cd-f36b3ce05360\") " pod="openshift-must-gather-nrxzw/must-gather-5k4zj" Jul 07 00:30:59.054285 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:59.054223 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b067ae86-e310-467b-a0cd-f36b3ce05360-must-gather-output\") pod \"must-gather-5k4zj\" (UID: \"b067ae86-e310-467b-a0cd-f36b3ce05360\") " pod="openshift-must-gather-nrxzw/must-gather-5k4zj" Jul 07 00:30:59.064868 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:59.064841 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n77lc\" (UniqueName: \"kubernetes.io/projected/b067ae86-e310-467b-a0cd-f36b3ce05360-kube-api-access-n77lc\") pod \"must-gather-5k4zj\" (UID: \"b067ae86-e310-467b-a0cd-f36b3ce05360\") " pod="openshift-must-gather-nrxzw/must-gather-5k4zj" Jul 07 00:30:59.120235 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:59.120185 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nrxzw/must-gather-5k4zj" Jul 07 00:30:59.267279 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:59.267189 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nrxzw/must-gather-5k4zj"] Jul 07 00:30:59.270208 ip-10-0-136-25 kubenswrapper[2553]: W0707 00:30:59.270180 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb067ae86_e310_467b_a0cd_f36b3ce05360.slice/crio-dbd39713dbe7afc84b1df52ad27fff9cb2abbeef68ff9eaf8466bac4c32505c1 WatchSource:0}: Error finding container dbd39713dbe7afc84b1df52ad27fff9cb2abbeef68ff9eaf8466bac4c32505c1: Status 404 returned error can't find the container with id dbd39713dbe7afc84b1df52ad27fff9cb2abbeef68ff9eaf8466bac4c32505c1 Jul 07 00:30:59.272021 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:30:59.272003 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 00:31:00.205491 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:00.205459 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nrxzw/must-gather-5k4zj" event={"ID":"b067ae86-e310-467b-a0cd-f36b3ce05360","Type":"ContainerStarted","Data":"dbd39713dbe7afc84b1df52ad27fff9cb2abbeef68ff9eaf8466bac4c32505c1"} Jul 07 00:31:01.216058 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:01.216015 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nrxzw/must-gather-5k4zj" event={"ID":"b067ae86-e310-467b-a0cd-f36b3ce05360","Type":"ContainerStarted","Data":"75ba07e69e4c197b412ef0af3c8951df73d96545f83683cbfcceb593a72ffe9c"} Jul 07 00:31:01.216717 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:01.216674 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nrxzw/must-gather-5k4zj" event={"ID":"b067ae86-e310-467b-a0cd-f36b3ce05360","Type":"ContainerStarted","Data":"d054eaa92c78a9c5b8d3a1401412b4e14a7cd764d2a46fb6ef354ee5285fb034"} Jul 07 00:31:01.236783 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:01.236732 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nrxzw/must-gather-5k4zj" podStartSLOduration=2.367816504 podStartE2EDuration="3.236716912s" totalImagesPullingTime="868.900408ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:30:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:31:01.234134898 +0000 UTC m=+2543.443864420" watchObservedRunningTime="2026-07-07 00:31:01.236716912 +0000 UTC m=+2543.446446432" Jul 07 00:31:01.669592 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:01.669562 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-55rgj_68c477a5-72a1-47ec-9864-17cd0021a5f9/global-pull-secret-syncer/0.log" Jul 07 00:31:01.873320 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:01.873274 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-j58wh_ff6ba6b4-93ca-481f-8193-dfedf07c3875/konnectivity-agent/0.log" Jul 07 00:31:01.939166 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:01.939078 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-25.ec2.internal_ba668ff1926990fccf3f73e0625e305c/haproxy/0.log" Jul 07 00:31:05.715006 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:05.714965 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c_d4edb7a7-2478-43e1-b346-d786e1606487/extract/0.log" Jul 07 00:31:05.738477 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:05.738445 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c_d4edb7a7-2478-43e1-b346-d786e1606487/util/0.log" Jul 07 00:31:05.766432 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:05.766369 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmq5c_d4edb7a7-2478-43e1-b346-d786e1606487/pull/0.log" Jul 07 00:31:05.799547 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:05.799522 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm_c4be0703-a52e-49d4-8942-edbb447df0ad/extract/0.log" Jul 07 00:31:05.822928 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:05.822903 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm_c4be0703-a52e-49d4-8942-edbb447df0ad/util/0.log" Jul 07 00:31:05.848350 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:05.848284 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xhmgm_c4be0703-a52e-49d4-8942-edbb447df0ad/pull/0.log" Jul 07 00:31:05.878816 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:05.878451 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s_7b1c8289-6786-41c6-a0fd-66e613080735/extract/0.log" Jul 07 00:31:05.901052 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:05.901010 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s_7b1c8289-6786-41c6-a0fd-66e613080735/util/0.log" Jul 07 00:31:05.925756 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:05.925723 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m5p8s_7b1c8289-6786-41c6-a0fd-66e613080735/pull/0.log" Jul 07 00:31:05.954729 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:05.954691 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v_a3ced335-f964-46a9-b311-969a96fc1136/extract/0.log" Jul 07 00:31:05.977311 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:05.977187 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v_a3ced335-f964-46a9-b311-969a96fc1136/util/0.log" Jul 07 00:31:06.003782 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:06.003748 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cfb5v_a3ced335-f964-46a9-b311-969a96fc1136/pull/0.log" Jul 07 00:31:06.552124 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:06.552056 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-xlhnc_a58195c7-3ab7-4748-a16f-58f9f6d1d23e/manager/0.log" Jul 07 00:31:06.594602 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:06.594559 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-sw76w_e090dac8-d5d5-4ea9-a0c2-1ffec011d9e2/manager/0.log" Jul 07 00:31:06.683628 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:06.683590 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-pmjrz_8708739e-7b51-4394-9e97-89a8c1b87997/registry-server/0.log" Jul 07 00:31:06.870966 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:06.870350 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-7pmgv_bf17cf71-15c2-40b4-81bb-df10ca03011c/manager/0.log" Jul 07 00:31:07.010428 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:07.006041 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-d8fs6_ee864a87-f13d-418c-9f4e-43dc12ce6add/manager/0.log" Jul 07 00:31:08.816176 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:08.816135 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-5m4nb_450852fa-55ab-4354-8781-e67dd3330970/cluster-monitoring-operator/0.log" Jul 07 00:31:08.846568 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:08.846522 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-kxl2w_4b569175-860d-44bc-841d-a268294a112d/kube-state-metrics/0.log" Jul 07 00:31:08.869754 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:08.869730 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-kxl2w_4b569175-860d-44bc-841d-a268294a112d/kube-rbac-proxy-main/0.log" Jul 07 00:31:08.894813 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:08.894776 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-kxl2w_4b569175-860d-44bc-841d-a268294a112d/kube-rbac-proxy-self/0.log" Jul 07 00:31:08.922364 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:08.922333 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-f6cf5fb68-c5vnf_503b39e8-97e2-4ad8-a4c5-0a716cd2061a/metrics-server/0.log" Jul 07 00:31:08.952728 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:08.952683 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-rtj7l_229ece0b-6bfb-4628-bb44-91087ba93096/monitoring-plugin/0.log" Jul 07 00:31:09.148219 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:09.148188 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xg2fq_376d341d-f1b9-4d45-90e8-4e77fe4f9837/node-exporter/0.log" Jul 07 00:31:09.176238 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:09.176206 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xg2fq_376d341d-f1b9-4d45-90e8-4e77fe4f9837/kube-rbac-proxy/0.log" Jul 07 00:31:09.201396 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:09.201364 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xg2fq_376d341d-f1b9-4d45-90e8-4e77fe4f9837/init-textfile/0.log" Jul 07 00:31:09.543139 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:09.543104 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-xswqw_45b28db1-4de7-47eb-83b3-5f728373814f/prometheus-operator/0.log" Jul 07 00:31:09.561636 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:09.561601 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-xswqw_45b28db1-4de7-47eb-83b3-5f728373814f/kube-rbac-proxy/0.log" Jul 07 00:31:09.600883 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:09.600800 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-pgkjd_22edb4ea-f244-4192-af32-adbea1984765/prometheus-operator-admission-webhook/0.log" Jul 07 00:31:12.343237 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:12.343200 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-76d6c4db4b-qvwbm_61611271-9dd7-4ef2-ab0c-9367e8cee39e/console/0.log" Jul 07 00:31:12.395186 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:12.395153 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-bvlqn_a5c9b8dc-3d47-439f-89bf-01f9077f4043/download-server/0.log" Jul 07 00:31:12.464106 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:12.463973 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nrxzw/perf-node-gather-daemonset-kxr8x"] Jul 07 00:31:12.472598 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:12.472570 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-kxr8x" Jul 07 00:31:12.477538 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:12.477433 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nrxzw/perf-node-gather-daemonset-kxr8x"] Jul 07 00:31:12.608932 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:12.608847 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f57b4a30-d6de-42d6-8f29-9e76c6715cdf-sys\") pod \"perf-node-gather-daemonset-kxr8x\" (UID: \"f57b4a30-d6de-42d6-8f29-9e76c6715cdf\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-kxr8x" Jul 07 00:31:12.608932 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:12.608899 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f57b4a30-d6de-42d6-8f29-9e76c6715cdf-podres\") pod \"perf-node-gather-daemonset-kxr8x\" (UID: \"f57b4a30-d6de-42d6-8f29-9e76c6715cdf\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-kxr8x" Jul 07 00:31:12.609156 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:12.608949 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccqjx\" (UniqueName: \"kubernetes.io/projected/f57b4a30-d6de-42d6-8f29-9e76c6715cdf-kube-api-access-ccqjx\") pod \"perf-node-gather-daemonset-kxr8x\" (UID: \"f57b4a30-d6de-42d6-8f29-9e76c6715cdf\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-kxr8x" Jul 07 00:31:12.609156 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:12.608986 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f57b4a30-d6de-42d6-8f29-9e76c6715cdf-proc\") pod \"perf-node-gather-daemonset-kxr8x\" (UID: \"f57b4a30-d6de-42d6-8f29-9e76c6715cdf\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-kxr8x" Jul 07 00:31:12.609156 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:12.609068 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f57b4a30-d6de-42d6-8f29-9e76c6715cdf-lib-modules\") pod \"perf-node-gather-daemonset-kxr8x\" (UID: \"f57b4a30-d6de-42d6-8f29-9e76c6715cdf\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-kxr8x" Jul 07 00:31:12.710009 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:12.709966 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f57b4a30-d6de-42d6-8f29-9e76c6715cdf-lib-modules\") pod \"perf-node-gather-daemonset-kxr8x\" (UID: \"f57b4a30-d6de-42d6-8f29-9e76c6715cdf\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-kxr8x" Jul 07 00:31:12.710182 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:12.710065 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f57b4a30-d6de-42d6-8f29-9e76c6715cdf-sys\") pod \"perf-node-gather-daemonset-kxr8x\" (UID: \"f57b4a30-d6de-42d6-8f29-9e76c6715cdf\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-kxr8x" Jul 07 00:31:12.710182 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:12.710098 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f57b4a30-d6de-42d6-8f29-9e76c6715cdf-podres\") pod \"perf-node-gather-daemonset-kxr8x\" (UID: \"f57b4a30-d6de-42d6-8f29-9e76c6715cdf\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-kxr8x" Jul 07 00:31:12.710182 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:12.710152 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccqjx\" (UniqueName: \"kubernetes.io/projected/f57b4a30-d6de-42d6-8f29-9e76c6715cdf-kube-api-access-ccqjx\") pod \"perf-node-gather-daemonset-kxr8x\" (UID: \"f57b4a30-d6de-42d6-8f29-9e76c6715cdf\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-kxr8x" Jul 07 00:31:12.710329 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:12.710193 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f57b4a30-d6de-42d6-8f29-9e76c6715cdf-proc\") pod \"perf-node-gather-daemonset-kxr8x\" (UID: \"f57b4a30-d6de-42d6-8f29-9e76c6715cdf\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-kxr8x" Jul 07 00:31:12.710392 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:12.710372 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f57b4a30-d6de-42d6-8f29-9e76c6715cdf-proc\") pod \"perf-node-gather-daemonset-kxr8x\" (UID: \"f57b4a30-d6de-42d6-8f29-9e76c6715cdf\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-kxr8x" Jul 07 00:31:12.710536 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:12.710516 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f57b4a30-d6de-42d6-8f29-9e76c6715cdf-lib-modules\") pod \"perf-node-gather-daemonset-kxr8x\" (UID: \"f57b4a30-d6de-42d6-8f29-9e76c6715cdf\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-kxr8x" Jul 07 00:31:12.710608 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:12.710577 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f57b4a30-d6de-42d6-8f29-9e76c6715cdf-sys\") pod \"perf-node-gather-daemonset-kxr8x\" (UID: \"f57b4a30-d6de-42d6-8f29-9e76c6715cdf\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-kxr8x" Jul 07 00:31:12.710676 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:12.710654 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f57b4a30-d6de-42d6-8f29-9e76c6715cdf-podres\") pod \"perf-node-gather-daemonset-kxr8x\" (UID: \"f57b4a30-d6de-42d6-8f29-9e76c6715cdf\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-kxr8x" Jul 07 00:31:12.720371 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:12.720340 2553 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccqjx\" (UniqueName: \"kubernetes.io/projected/f57b4a30-d6de-42d6-8f29-9e76c6715cdf-kube-api-access-ccqjx\") pod \"perf-node-gather-daemonset-kxr8x\" (UID: \"f57b4a30-d6de-42d6-8f29-9e76c6715cdf\") " pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-kxr8x" Jul 07 00:31:12.792065 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:12.792031 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-kxr8x" Jul 07 00:31:12.972500 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:12.972464 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nrxzw/perf-node-gather-daemonset-kxr8x"] Jul 07 00:31:13.358658 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:13.357828 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-kxr8x" event={"ID":"f57b4a30-d6de-42d6-8f29-9e76c6715cdf","Type":"ContainerStarted","Data":"5f13713e23181b2fd3e37c7ee3f8e40c98828a53541085302152120920d42676"} Jul 07 00:31:13.358658 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:13.357899 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-kxr8x" event={"ID":"f57b4a30-d6de-42d6-8f29-9e76c6715cdf","Type":"ContainerStarted","Data":"028fc719643355401aeea2e8dcdeab382c40aed203d2bbfe6f0aefc68d1a87b8"} Jul 07 00:31:13.358658 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:13.358149 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-kxr8x" Jul 07 00:31:13.377486 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:13.377430 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-kxr8x" podStartSLOduration=1.377415735 podStartE2EDuration="1.377415735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 00:31:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:31:13.375162064 +0000 UTC m=+2555.584891586" watchObservedRunningTime="2026-07-07 00:31:13.377415735 +0000 UTC m=+2555.587145255" Jul 07 00:31:14.208554 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:14.208525 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v978g_05a76f14-b4fd-4087-bd7a-53587b38ea81/dns/0.log" Jul 07 00:31:14.229713 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:14.229675 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v978g_05a76f14-b4fd-4087-bd7a-53587b38ea81/kube-rbac-proxy/0.log" Jul 07 00:31:14.267268 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:14.267238 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9tms8_93308370-a961-4b85-a764-abac1d17c520/dns-node-resolver/0.log" Jul 07 00:31:14.964738 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:14.964703 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-pruner-29723040-zbd4s_84a468e6-58a8-4f68-8a2b-929c319ec7e0/image-pruner/0.log" Jul 07 00:31:15.076331 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:15.076277 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tdk4b_4f2c8799-a48e-4f4c-bce4-7f0187c24298/node-ca/0.log" Jul 07 00:31:15.989446 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:15.989411 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678sppw9_379fe0e0-dc84-4b69-a99d-dcf861e32ce6/istio-proxy/0.log" Jul 07 00:31:16.120011 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:16.119980 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dc8c455ff-sbm28_b94f94fd-f309-4495-86af-fb0eb3946b4f/kube-auth-proxy/0.log" Jul 07 00:31:16.175706 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:16.175679 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-7x84r_867fe4ad-91d7-4301-b82a-f5eaf31f863d/istio-proxy/0.log" Jul 07 00:31:16.212632 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:16.212597 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-7x84r_867fe4ad-91d7-4301-b82a-f5eaf31f863d/istio-proxy/1.log" Jul 07 00:31:16.249228 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:16.249130 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-tmjqm_c6582771-54aa-4c55-b416-3f661db37add/payload-pre-processing/0.log" Jul 07 00:31:16.306279 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:16.306250 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-2bhlm_e9cf329f-b667-4b93-8511-811c1f1efee3/payload-processing/0.log" Jul 07 00:31:16.995386 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:16.995346 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7gntn_e7b30a60-5ac4-43d8-891c-d23dd770b332/serve-healthcheck-canary/0.log" Jul 07 00:31:17.554020 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:17.553984 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rt6cm_44cf08e2-0939-4411-8f46-7407b0e66e33/kube-rbac-proxy/0.log" Jul 07 00:31:17.576391 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:17.576363 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rt6cm_44cf08e2-0939-4411-8f46-7407b0e66e33/exporter/0.log" Jul 07 00:31:17.599871 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:17.599841 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rt6cm_44cf08e2-0939-4411-8f46-7407b0e66e33/extractor/0.log" Jul 07 00:31:19.385321 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:19.385261 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nrxzw/perf-node-gather-daemonset-kxr8x" Jul 07 00:31:19.995188 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:19.995154 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-szrs8_6df592ff-ca58-4349-9af2-0d2175c2e995/manager/0.log" Jul 07 00:31:20.172283 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:20.172241 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-bwhj6_e4805476-0013-484a-b046-999b6308296f/manager/0.log" Jul 07 00:31:20.213425 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:20.213394 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-7bdf95f944-bdxls_818a2117-421b-478b-9633-a95a0a927158/maas-api/0.log" Jul 07 00:31:20.235184 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:20.235161 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723040-s84z8_34d89dc6-76e4-42c1-846a-6708ea0d87ea/cleanup/0.log" Jul 07 00:31:20.412246 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:20.412214 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-56bf7f957f-sjkb7_52bd584a-8a0e-462f-9ae5-79c5295eab77/manager/0.log" Jul 07 00:31:20.738513 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:20.738390 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bcdcfc745-xcv6b_0209cf45-9d4a-4f99-8cdb-29e120237003/manager/0.log" Jul 07 00:31:20.757578 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:20.757546 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bcdcfc745-xcv6b_0209cf45-9d4a-4f99-8cdb-29e120237003/copy-manifests/0.log" Jul 07 00:31:20.782038 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:20.782007 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-mjkm4_d27d869f-2a23-476d-9006-3e5c71823782/postgres/0.log" Jul 07 00:31:22.650017 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:22.649940 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678sppw9_379fe0e0-dc84-4b69-a99d-dcf861e32ce6/istio-proxy/0.log" Jul 07 00:31:22.759824 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:22.759800 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dc8c455ff-sbm28_b94f94fd-f309-4495-86af-fb0eb3946b4f/kube-auth-proxy/0.log" Jul 07 00:31:22.796874 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:22.796840 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-7x84r_867fe4ad-91d7-4301-b82a-f5eaf31f863d/istio-proxy/0.log" Jul 07 00:31:22.830429 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:22.830384 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-7x84r_867fe4ad-91d7-4301-b82a-f5eaf31f863d/istio-proxy/1.log" Jul 07 00:31:22.854989 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:22.854956 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-tmjqm_c6582771-54aa-4c55-b416-3f661db37add/payload-pre-processing/0.log" Jul 07 00:31:22.900209 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:22.900108 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-2bhlm_e9cf329f-b667-4b93-8511-811c1f1efee3/payload-processing/0.log" Jul 07 00:31:24.063256 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:24.063145 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678sppw9_379fe0e0-dc84-4b69-a99d-dcf861e32ce6/istio-proxy/0.log" Jul 07 00:31:24.179141 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:24.179115 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5dc8c455ff-sbm28_b94f94fd-f309-4495-86af-fb0eb3946b4f/kube-auth-proxy/0.log" Jul 07 00:31:24.212475 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:24.212449 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-7x84r_867fe4ad-91d7-4301-b82a-f5eaf31f863d/istio-proxy/0.log" Jul 07 00:31:24.252506 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:24.252479 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-7x84r_867fe4ad-91d7-4301-b82a-f5eaf31f863d/istio-proxy/1.log" Jul 07 00:31:24.278788 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:24.278744 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-tmjqm_c6582771-54aa-4c55-b416-3f661db37add/payload-pre-processing/0.log" Jul 07 00:31:24.329052 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:24.328964 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-2bhlm_e9cf329f-b667-4b93-8511-811c1f1efee3/payload-processing/0.log" Jul 07 00:31:28.601212 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:28.601181 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6r2vp_43908b50-c882-4e43-8d91-107f2acaf7b5/kube-multus-additional-cni-plugins/0.log" Jul 07 00:31:28.628449 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:28.628420 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6r2vp_43908b50-c882-4e43-8d91-107f2acaf7b5/egress-router-binary-copy/0.log" Jul 07 00:31:28.650641 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:28.650609 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6r2vp_43908b50-c882-4e43-8d91-107f2acaf7b5/cni-plugins/0.log" Jul 07 00:31:28.671336 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:28.671307 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6r2vp_43908b50-c882-4e43-8d91-107f2acaf7b5/bond-cni-plugin/0.log" Jul 07 00:31:28.692469 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:28.692437 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6r2vp_43908b50-c882-4e43-8d91-107f2acaf7b5/routeoverride-cni/0.log" Jul 07 00:31:28.712881 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:28.712852 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6r2vp_43908b50-c882-4e43-8d91-107f2acaf7b5/whereabouts-cni-bincopy/0.log" Jul 07 00:31:28.734551 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:28.734404 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6r2vp_43908b50-c882-4e43-8d91-107f2acaf7b5/whereabouts-cni/0.log" Jul 07 00:31:29.118029 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:29.118003 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-p274v_ff9e3a5f-ea1a-4715-87b1-2766a82bef28/kube-multus/0.log" Jul 07 00:31:29.264815 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:29.264790 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-zmkd8_971187a6-5709-48db-b990-0a13db9fad13/network-metrics-daemon/0.log" Jul 07 00:31:29.284639 ip-10-0-136-25 kubenswrapper[2553]: I0707 00:31:29.284612 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-zmkd8_971187a6-5709-48db-b990-0a13db9fad13/kube-rbac-proxy/0.log"