Jul 06 18:19:43.151173 ip-10-0-128-98 systemd[1]: Starting Kubernetes Kubelet... Jul 06 18:19:43.583127 ip-10-0-128-98 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 18:19:43.583127 ip-10-0-128-98 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 18:19:43.583127 ip-10-0-128-98 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 18:19:43.583127 ip-10-0-128-98 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 18:19:43.583127 ip-10-0-128-98 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 18:19:43.583760 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.583140 2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 18:19:43.588492 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588471 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 18:19:43.588492 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588490 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 18:19:43.588492 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588494 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 18:19:43.588585 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588497 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 18:19:43.588585 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588500 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 18:19:43.588585 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588504 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 18:19:43.588585 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588507 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 18:19:43.588585 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588509 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 18:19:43.588585 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588512 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 18:19:43.588585 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588515 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 18:19:43.588585 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588518 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 18:19:43.588585 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588520 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 18:19:43.588585 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588523 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 18:19:43.588585 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588525 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 18:19:43.588585 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588529 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 18:19:43.588585 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588532 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 18:19:43.588585 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588534 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 18:19:43.588585 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588537 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 18:19:43.588585 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588540 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 18:19:43.588585 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588542 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 18:19:43.588585 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588545 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 18:19:43.588585 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588547 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 18:19:43.588585 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588550 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 18:19:43.589091 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588552 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 18:19:43.589091 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588555 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 18:19:43.589091 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588557 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 18:19:43.589091 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588559 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 18:19:43.589091 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588562 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 18:19:43.589091 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588564 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 18:19:43.589091 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588567 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 18:19:43.589091 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588569 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 18:19:43.589091 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588571 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 18:19:43.589091 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588574 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 18:19:43.589091 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588576 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 18:19:43.589091 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588579 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 18:19:43.589091 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588582 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 18:19:43.589091 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588585 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 18:19:43.589091 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588588 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 18:19:43.589091 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588591 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 18:19:43.589091 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588593 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 18:19:43.589091 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588596 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 18:19:43.589091 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588598 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 18:19:43.589091 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588601 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 18:19:43.589612 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588603 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 18:19:43.589612 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588606 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 18:19:43.589612 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588609 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 18:19:43.589612 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588611 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 18:19:43.589612 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588614 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 18:19:43.589612 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588617 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 18:19:43.589612 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588619 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 18:19:43.589612 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588621 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 18:19:43.589612 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588624 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 18:19:43.589612 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588626 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 18:19:43.589612 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588628 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 18:19:43.589612 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588631 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 18:19:43.589612 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588633 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 18:19:43.589612 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588636 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 18:19:43.589612 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588638 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 18:19:43.589612 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588642 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 18:19:43.589612 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588645 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 18:19:43.589612 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588649 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 18:19:43.589612 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588652 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 18:19:43.590088 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588655 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 18:19:43.590088 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588658 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 18:19:43.590088 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588661 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 18:19:43.590088 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588663 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 18:19:43.590088 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588667 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 18:19:43.590088 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588671 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 18:19:43.590088 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588676 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 18:19:43.590088 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588679 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 18:19:43.590088 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588703 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 18:19:43.590088 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588708 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 18:19:43.590088 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588711 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 18:19:43.590088 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588714 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 18:19:43.590088 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588717 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 18:19:43.590088 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588720 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 18:19:43.590088 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588722 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 18:19:43.590088 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588725 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 18:19:43.590088 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588728 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 18:19:43.590088 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588731 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 18:19:43.590088 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588735 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 18:19:43.590548 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588739 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 18:19:43.590548 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588742 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 18:19:43.590548 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588744 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 18:19:43.590548 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588747 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 18:19:43.590548 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588750 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 06 18:19:43.590548 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588752 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 18:19:43.590548 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588755 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 18:19:43.590548 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588758 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 18:19:43.590548 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588761 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 18:19:43.590548 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588763 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 18:19:43.590548 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588766 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 18:19:43.590548 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588769 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 18:19:43.590548 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588772 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 18:19:43.590548 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588775 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 18:19:43.590548 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588777 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 18:19:43.590548 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588780 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 18:19:43.590548 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588782 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 18:19:43.590548 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.588784 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 18:19:43.590990 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.590939 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 18:19:43.590990 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.590947 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 18:19:43.590990 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.590952 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 18:19:43.590990 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.590955 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 18:19:43.590990 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.590958 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 18:19:43.590990 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.590961 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 18:19:43.590990 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.590963 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 18:19:43.590990 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.590966 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 18:19:43.590990 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.590969 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 18:19:43.590990 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.590971 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 18:19:43.590990 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.590973 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 18:19:43.590990 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.590976 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 18:19:43.590990 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.590979 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 18:19:43.590990 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.590983 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 18:19:43.590990 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.590988 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 18:19:43.590990 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.590990 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 18:19:43.590990 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.590993 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 18:19:43.590990 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.590996 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 18:19:43.590990 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.590999 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 18:19:43.591471 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591002 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 18:19:43.591471 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591005 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 18:19:43.591471 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591007 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 18:19:43.591471 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591010 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 18:19:43.591471 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591012 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 18:19:43.591471 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591015 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 18:19:43.591471 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591017 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 18:19:43.591471 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591019 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 18:19:43.591471 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591022 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 18:19:43.591471 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591024 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 18:19:43.591471 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591028 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 18:19:43.591471 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591030 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 18:19:43.591471 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591033 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 18:19:43.591471 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591035 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 18:19:43.591471 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591038 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 18:19:43.591471 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591040 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 18:19:43.591471 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591043 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 18:19:43.591471 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591045 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 18:19:43.591471 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591047 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 18:19:43.591471 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591050 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 18:19:43.592011 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591052 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 18:19:43.592011 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591055 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 18:19:43.592011 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591057 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 18:19:43.592011 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591059 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 18:19:43.592011 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591062 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 18:19:43.592011 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591064 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 18:19:43.592011 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591067 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 18:19:43.592011 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591069 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 18:19:43.592011 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591072 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 18:19:43.592011 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591075 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 18:19:43.592011 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591077 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 18:19:43.592011 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591079 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 18:19:43.592011 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591082 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 18:19:43.592011 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591085 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 18:19:43.592011 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591088 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 18:19:43.592011 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591090 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 18:19:43.592011 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591093 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 18:19:43.592011 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591095 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 18:19:43.592011 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591098 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 06 18:19:43.592011 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591100 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 18:19:43.592500 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591103 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 18:19:43.592500 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591105 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 18:19:43.592500 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591107 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 18:19:43.592500 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591110 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 18:19:43.592500 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591112 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 18:19:43.592500 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591115 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 18:19:43.592500 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591117 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 18:19:43.592500 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591120 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 18:19:43.592500 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591122 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 18:19:43.592500 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591125 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 18:19:43.592500 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591127 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 18:19:43.592500 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591130 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 18:19:43.592500 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591139 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 18:19:43.592500 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591145 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 18:19:43.592500 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591148 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 18:19:43.592500 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591152 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 18:19:43.592500 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591154 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 18:19:43.592500 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591157 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 18:19:43.592500 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591161 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 18:19:43.593156 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591164 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 18:19:43.593156 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591167 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 18:19:43.593156 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591171 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 18:19:43.593156 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591174 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 18:19:43.593156 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591176 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 18:19:43.593156 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591179 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 18:19:43.593156 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591184 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 18:19:43.593156 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591187 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 18:19:43.593156 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591190 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 18:19:43.593156 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591193 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 18:19:43.593156 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591196 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 18:19:43.593156 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591198 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 18:19:43.593156 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591201 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 18:19:43.593156 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591203 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 18:19:43.593156 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591206 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 18:19:43.593156 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591208 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 18:19:43.593156 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591210 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 18:19:43.593156 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591212 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 18:19:43.593156 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591215 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 18:19:43.593156 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591217 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 18:19:43.593936 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591220 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 18:19:43.593936 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591637 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 18:19:43.593936 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591643 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 18:19:43.593936 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591646 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 18:19:43.593936 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591649 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 18:19:43.593936 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591652 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 18:19:43.593936 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591655 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 18:19:43.593936 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591657 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 18:19:43.593936 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591660 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 18:19:43.593936 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591663 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 18:19:43.593936 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591666 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 18:19:43.593936 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591668 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 18:19:43.593936 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591671 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 18:19:43.593936 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591673 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 18:19:43.593936 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591677 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 18:19:43.593936 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591679 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 18:19:43.593936 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591682 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 18:19:43.593936 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591684 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 18:19:43.593936 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591687 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 18:19:43.593936 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591690 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 18:19:43.594453 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591692 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 18:19:43.594453 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591695 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 18:19:43.594453 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591698 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 18:19:43.594453 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591700 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 18:19:43.594453 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591703 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 18:19:43.594453 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591705 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 18:19:43.594453 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591708 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 18:19:43.594453 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591710 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 18:19:43.594453 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591713 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 18:19:43.594453 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591715 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 18:19:43.594453 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591718 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 18:19:43.594453 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591720 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 18:19:43.594453 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591722 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 18:19:43.594453 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591725 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 18:19:43.594453 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591727 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 18:19:43.594453 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591730 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 18:19:43.594453 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591732 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 18:19:43.594453 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591735 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 18:19:43.594453 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591737 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 18:19:43.594934 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591741 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 18:19:43.594934 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591745 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 18:19:43.594934 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591748 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 18:19:43.594934 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591750 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 18:19:43.594934 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591753 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 18:19:43.594934 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591755 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 18:19:43.594934 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591758 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 18:19:43.594934 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591760 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 18:19:43.594934 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591763 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 18:19:43.594934 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591766 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 18:19:43.594934 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591768 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 18:19:43.594934 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591771 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 18:19:43.594934 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591774 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 18:19:43.594934 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591776 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 18:19:43.594934 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591779 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 18:19:43.594934 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591781 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 18:19:43.594934 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591783 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 18:19:43.594934 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591786 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 18:19:43.594934 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591788 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 18:19:43.595411 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591791 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 18:19:43.595411 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591793 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 18:19:43.595411 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591795 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 18:19:43.595411 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591798 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 18:19:43.595411 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591800 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 18:19:43.595411 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591802 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 18:19:43.595411 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591805 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 06 18:19:43.595411 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591807 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 18:19:43.595411 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591810 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 18:19:43.595411 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591812 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 18:19:43.595411 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591815 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 18:19:43.595411 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591819 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 18:19:43.595411 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591822 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 18:19:43.595411 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591825 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 18:19:43.595411 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591828 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 18:19:43.595411 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591830 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 18:19:43.595411 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591833 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 18:19:43.595411 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591835 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 18:19:43.595411 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591838 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 18:19:43.595411 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591840 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 18:19:43.595894 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591842 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 18:19:43.595894 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591844 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 18:19:43.595894 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591847 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 18:19:43.595894 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591851 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 18:19:43.595894 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591854 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 18:19:43.595894 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591857 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 18:19:43.595894 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591860 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 18:19:43.595894 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591862 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 18:19:43.595894 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591882 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 18:19:43.595894 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591885 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 18:19:43.595894 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591887 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 18:19:43.595894 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591890 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 18:19:43.595894 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591892 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 18:19:43.595894 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591895 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 18:19:43.595894 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591898 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 18:19:43.595894 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591900 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 18:19:43.595894 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591903 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 18:19:43.595894 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591905 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 18:19:43.595894 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591908 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 18:19:43.595894 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591910 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 18:19:43.596473 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591913 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 18:19:43.596473 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.591915 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 18:19:43.596473 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.591993 2566 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 18:19:43.596473 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592008 2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 18:19:43.596473 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592015 2566 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 18:19:43.596473 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592021 2566 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 18:19:43.596473 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592025 2566 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 18:19:43.596473 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592028 2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 18:19:43.596473 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592032 2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 18:19:43.596473 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592036 2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 18:19:43.596473 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592040 2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 18:19:43.596473 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592042 2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 18:19:43.596473 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592046 2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 18:19:43.596473 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592049 2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 18:19:43.596473 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592052 2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 18:19:43.596473 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592056 2566 flags.go:64] FLAG: --cgroup-root="" Jul 06 18:19:43.596473 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592059 2566 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 18:19:43.596473 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592062 2566 flags.go:64] FLAG: --client-ca-file="" Jul 06 18:19:43.596473 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592065 2566 flags.go:64] FLAG: --cloud-provider="external" Jul 06 18:19:43.596473 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592068 2566 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 18:19:43.596473 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592073 2566 flags.go:64] FLAG: --cluster-domain="" Jul 06 18:19:43.596473 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592076 2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 18:19:43.596473 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592079 2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 18:19:43.597046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592082 2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 18:19:43.597046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592086 2566 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 18:19:43.597046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592090 2566 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 18:19:43.597046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592093 2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 18:19:43.597046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592096 2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 18:19:43.597046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592099 2566 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 18:19:43.597046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592102 2566 flags.go:64] FLAG: --contention-profiling="false" Jul 06 18:19:43.597046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592105 2566 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 18:19:43.597046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592108 2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 18:19:43.597046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592111 2566 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 18:19:43.597046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592113 2566 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 18:19:43.597046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592117 2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 18:19:43.597046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592120 2566 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 18:19:43.597046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592123 2566 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 18:19:43.597046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592126 2566 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 18:19:43.597046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592129 2566 flags.go:64] FLAG: --enable-server="true" Jul 06 18:19:43.597046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592132 2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 18:19:43.597046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592136 2566 flags.go:64] FLAG: --event-burst="100" Jul 06 18:19:43.597046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592139 2566 flags.go:64] FLAG: --event-qps="50" Jul 06 18:19:43.597046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592142 2566 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 18:19:43.597046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592145 2566 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 18:19:43.597046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592148 2566 flags.go:64] FLAG: --eviction-hard="" Jul 06 18:19:43.597046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592152 2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 18:19:43.597046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592154 2566 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 18:19:43.597612 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592157 2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 18:19:43.597612 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592162 2566 flags.go:64] FLAG: --eviction-soft="" Jul 06 18:19:43.597612 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592165 2566 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 18:19:43.597612 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592168 2566 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 18:19:43.597612 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592171 2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 18:19:43.597612 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592174 2566 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 18:19:43.597612 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592177 2566 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 18:19:43.597612 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592180 2566 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 18:19:43.597612 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592182 2566 flags.go:64] FLAG: --feature-gates="" Jul 06 18:19:43.597612 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592186 2566 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 18:19:43.597612 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592189 2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 18:19:43.597612 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592192 2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 18:19:43.597612 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592195 2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 18:19:43.597612 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592198 2566 flags.go:64] FLAG: --healthz-port="10248" Jul 06 18:19:43.597612 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592201 2566 flags.go:64] FLAG: --help="false" Jul 06 18:19:43.597612 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592204 2566 flags.go:64] FLAG: --hostname-override="ip-10-0-128-98.ec2.internal" Jul 06 18:19:43.597612 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592207 2566 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 18:19:43.597612 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592210 2566 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 18:19:43.597612 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592212 2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 18:19:43.597612 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592216 2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 18:19:43.597612 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592219 2566 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 18:19:43.597612 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592222 2566 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 18:19:43.597612 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592225 2566 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 18:19:43.598251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592228 2566 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 18:19:43.598251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592231 2566 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 18:19:43.598251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592233 2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 18:19:43.598251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592236 2566 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 18:19:43.598251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592239 2566 flags.go:64] FLAG: --kube-reserved="" Jul 06 18:19:43.598251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592242 2566 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 18:19:43.598251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592244 2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 18:19:43.598251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592247 2566 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 18:19:43.598251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592250 2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 18:19:43.598251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592253 2566 flags.go:64] FLAG: --lock-file="" Jul 06 18:19:43.598251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592258 2566 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 18:19:43.598251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592261 2566 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 18:19:43.598251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592264 2566 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 18:19:43.598251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592270 2566 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 18:19:43.598251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592273 2566 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 18:19:43.598251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592276 2566 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 18:19:43.598251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592278 2566 flags.go:64] FLAG: --logging-format="text" Jul 06 18:19:43.598251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592281 2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 18:19:43.598251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592298 2566 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 18:19:43.598251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592301 2566 flags.go:64] FLAG: --manifest-url="" Jul 06 18:19:43.598251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592305 2566 flags.go:64] FLAG: --manifest-url-header="" Jul 06 18:19:43.598251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592309 2566 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 18:19:43.598251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592312 2566 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 18:19:43.598251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592316 2566 flags.go:64] FLAG: --max-pods="110" Jul 06 18:19:43.598251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592319 2566 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 18:19:43.598858 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592322 2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 18:19:43.598858 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592325 2566 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 18:19:43.598858 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592328 2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 18:19:43.598858 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592333 2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 18:19:43.598858 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592336 2566 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 18:19:43.598858 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592339 2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 18:19:43.598858 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592346 2566 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 18:19:43.598858 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592349 2566 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 18:19:43.598858 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592352 2566 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 18:19:43.598858 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592355 2566 flags.go:64] FLAG: --pod-cidr="" Jul 06 18:19:43.598858 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592358 2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 18:19:43.598858 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592364 2566 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 18:19:43.598858 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592367 2566 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 18:19:43.598858 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592370 2566 flags.go:64] FLAG: --pods-per-core="0" Jul 06 18:19:43.598858 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592373 2566 flags.go:64] FLAG: --port="10250" Jul 06 18:19:43.598858 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592375 2566 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 18:19:43.598858 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592378 2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0cfaa014367ca1ac3" Jul 06 18:19:43.598858 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592382 2566 flags.go:64] FLAG: --qos-reserved="" Jul 06 18:19:43.598858 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592387 2566 flags.go:64] FLAG: --read-only-port="10255" Jul 06 18:19:43.598858 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592390 2566 flags.go:64] FLAG: --register-node="true" Jul 06 18:19:43.598858 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592393 2566 flags.go:64] FLAG: --register-with-taints="" Jul 06 18:19:43.598858 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592397 2566 flags.go:64] FLAG: --registry-burst="10" Jul 06 18:19:43.598858 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592400 2566 flags.go:64] FLAG: --registry-qps="5" Jul 06 18:19:43.598858 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592403 2566 flags.go:64] FLAG: --reserved-cpus="" Jul 06 18:19:43.599470 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592405 2566 flags.go:64] FLAG: --reserved-memory="" Jul 06 18:19:43.599470 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592409 2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 18:19:43.599470 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592412 2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 18:19:43.599470 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592414 2566 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 18:19:43.599470 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592417 2566 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 18:19:43.599470 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592420 2566 flags.go:64] FLAG: --runonce="false" Jul 06 18:19:43.599470 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592423 2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 18:19:43.599470 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592426 2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 18:19:43.599470 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592429 2566 flags.go:64] FLAG: --seccomp-default="false" Jul 06 18:19:43.599470 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592431 2566 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 18:19:43.599470 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592434 2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 18:19:43.599470 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592437 2566 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 18:19:43.599470 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592440 2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 18:19:43.599470 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592443 2566 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 18:19:43.599470 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592447 2566 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 18:19:43.599470 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592450 2566 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 18:19:43.599470 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592452 2566 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 18:19:43.599470 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592456 2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 18:19:43.599470 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592459 2566 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 18:19:43.599470 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592461 2566 flags.go:64] FLAG: --system-cgroups="" Jul 06 18:19:43.599470 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592464 2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 18:19:43.599470 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592469 2566 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 18:19:43.599470 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592472 2566 flags.go:64] FLAG: --tls-cert-file="" Jul 06 18:19:43.599470 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592474 2566 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 18:19:43.599470 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592478 2566 flags.go:64] FLAG: --tls-min-version="" Jul 06 18:19:43.600094 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592480 2566 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 18:19:43.600094 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592485 2566 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 18:19:43.600094 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592488 2566 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 18:19:43.600094 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592492 2566 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 18:19:43.600094 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592494 2566 flags.go:64] FLAG: --v="2" Jul 06 18:19:43.600094 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592498 2566 flags.go:64] FLAG: --version="false" Jul 06 18:19:43.600094 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592502 2566 flags.go:64] FLAG: --vmodule="" Jul 06 18:19:43.600094 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592506 2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 18:19:43.600094 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.592509 2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 18:19:43.600094 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593188 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 18:19:43.600094 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593200 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 18:19:43.600094 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593206 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 18:19:43.600094 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593211 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 18:19:43.600094 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593221 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 18:19:43.600094 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593225 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 18:19:43.600094 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593229 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 18:19:43.600094 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593233 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 18:19:43.600094 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593238 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 18:19:43.600094 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593242 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 18:19:43.600094 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593318 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 18:19:43.600094 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593326 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 18:19:43.600094 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593330 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 18:19:43.600618 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593334 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 18:19:43.600618 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593337 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 18:19:43.600618 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593340 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 18:19:43.600618 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593344 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 18:19:43.600618 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593347 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 18:19:43.600618 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593350 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 18:19:43.600618 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593352 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 18:19:43.600618 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593355 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 18:19:43.600618 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593358 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 18:19:43.600618 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593364 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 18:19:43.600618 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593367 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 18:19:43.600618 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593370 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 18:19:43.600618 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593372 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 18:19:43.600618 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593375 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 18:19:43.600618 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593380 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 18:19:43.600618 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593384 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 18:19:43.600618 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593388 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 18:19:43.600618 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593392 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 18:19:43.600618 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593396 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 18:19:43.601090 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593400 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 18:19:43.601090 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593405 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 06 18:19:43.601090 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593413 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 18:19:43.601090 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593417 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 18:19:43.601090 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593423 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 18:19:43.601090 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593426 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 18:19:43.601090 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593430 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 18:19:43.601090 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593434 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 18:19:43.601090 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593437 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 18:19:43.601090 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593441 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 18:19:43.601090 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593446 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 18:19:43.601090 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593452 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 18:19:43.601090 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593457 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 18:19:43.601090 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593462 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 18:19:43.601090 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593466 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 18:19:43.601090 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593475 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 18:19:43.601090 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593479 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 18:19:43.601090 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593484 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 18:19:43.601090 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593489 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 18:19:43.601090 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593493 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 18:19:43.601606 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593497 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 18:19:43.601606 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593501 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 18:19:43.601606 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593510 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 18:19:43.601606 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593515 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 18:19:43.601606 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593520 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 18:19:43.601606 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593524 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 18:19:43.601606 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593530 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 18:19:43.601606 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593539 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 18:19:43.601606 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593543 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 18:19:43.601606 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593548 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 18:19:43.601606 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593554 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 18:19:43.601606 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593560 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 18:19:43.601606 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593564 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 18:19:43.601606 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593569 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 18:19:43.601606 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593572 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 18:19:43.601606 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593577 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 18:19:43.601606 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593581 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 18:19:43.601606 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593585 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 18:19:43.601606 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593589 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 18:19:43.602109 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593598 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 18:19:43.602109 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593602 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 18:19:43.602109 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593607 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 18:19:43.602109 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593613 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 18:19:43.602109 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593618 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 18:19:43.602109 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593624 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 18:19:43.602109 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593628 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 18:19:43.602109 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593632 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 18:19:43.602109 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593636 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 18:19:43.602109 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593640 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 18:19:43.602109 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593644 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 18:19:43.602109 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593649 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 18:19:43.602109 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593653 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 18:19:43.602109 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593661 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 18:19:43.602109 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593665 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 18:19:43.602109 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593670 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 18:19:43.602109 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593674 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 18:19:43.602109 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593678 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 18:19:43.602109 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593682 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 18:19:43.602109 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593687 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 18:19:43.602600 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593692 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 18:19:43.602600 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593696 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 18:19:43.602600 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593701 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 18:19:43.602600 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593705 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 18:19:43.602600 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593711 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 18:19:43.602600 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593717 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 18:19:43.602600 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593726 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 18:19:43.602600 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.593731 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 18:19:43.602600 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.593739 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 18:19:43.602600 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.601054 2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 18:19:43.602600 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.601070 2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 18:19:43.602600 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.601083 2566 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 18:19:43.602600 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.601089 2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 18:19:43.602600 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601144 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 18:19:43.602970 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601149 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 18:19:43.602970 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601152 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 18:19:43.602970 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601155 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 18:19:43.602970 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601159 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 18:19:43.602970 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601162 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 18:19:43.602970 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601165 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 18:19:43.602970 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601168 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 18:19:43.602970 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601171 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 18:19:43.602970 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601174 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 18:19:43.602970 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601176 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 18:19:43.602970 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601179 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 18:19:43.602970 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601182 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 18:19:43.602970 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601184 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 18:19:43.602970 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601187 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 18:19:43.602970 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601190 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 18:19:43.602970 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601194 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 18:19:43.602970 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601197 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 18:19:43.602970 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601200 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 18:19:43.602970 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601204 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 18:19:43.603431 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601206 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 18:19:43.603431 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601209 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 18:19:43.603431 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601212 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 18:19:43.603431 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601214 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 18:19:43.603431 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601217 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 18:19:43.603431 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601220 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 18:19:43.603431 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601222 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 18:19:43.603431 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601225 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 18:19:43.603431 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601227 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 18:19:43.603431 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601230 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 18:19:43.603431 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601232 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 18:19:43.603431 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601235 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 18:19:43.603431 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601238 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 18:19:43.603431 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601240 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 18:19:43.603431 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601244 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 18:19:43.603431 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601246 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 18:19:43.603431 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601249 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 18:19:43.603431 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601251 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 18:19:43.603431 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601254 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 18:19:43.603431 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601257 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 18:19:43.603922 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601259 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 18:19:43.603922 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601262 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 18:19:43.603922 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601264 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 18:19:43.603922 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601267 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 18:19:43.603922 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601270 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 18:19:43.603922 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601272 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 18:19:43.603922 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601275 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 18:19:43.603922 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601277 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 18:19:43.603922 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601280 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 18:19:43.603922 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601282 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 18:19:43.603922 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601285 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 18:19:43.603922 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601287 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 18:19:43.603922 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601290 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 18:19:43.603922 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601292 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 18:19:43.603922 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601295 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 18:19:43.603922 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601297 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 18:19:43.603922 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601300 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 18:19:43.603922 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601302 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 18:19:43.603922 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601305 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 18:19:43.604392 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601307 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 18:19:43.604392 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601310 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 18:19:43.604392 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601312 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 06 18:19:43.604392 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601315 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 18:19:43.604392 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601318 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 18:19:43.604392 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601322 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 18:19:43.604392 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601326 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 18:19:43.604392 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601329 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 18:19:43.604392 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601332 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 18:19:43.604392 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601335 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 18:19:43.604392 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601338 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 18:19:43.604392 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601341 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 18:19:43.604392 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601344 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 18:19:43.604392 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601347 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 18:19:43.604392 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601350 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 18:19:43.604392 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601353 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 18:19:43.604392 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601357 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 18:19:43.604392 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601360 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 18:19:43.604392 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601363 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 18:19:43.604856 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601366 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 18:19:43.604856 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601370 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 18:19:43.604856 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601372 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 18:19:43.604856 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601375 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 18:19:43.604856 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601378 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 18:19:43.604856 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601381 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 18:19:43.604856 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601383 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 18:19:43.604856 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601386 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 18:19:43.604856 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601389 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 18:19:43.604856 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601392 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 18:19:43.604856 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601394 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 18:19:43.604856 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601397 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 18:19:43.604856 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601399 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 18:19:43.604856 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601402 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 18:19:43.604856 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601404 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 18:19:43.604856 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601407 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 18:19:43.604856 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601409 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 18:19:43.604856 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601412 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 18:19:43.604856 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601415 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 18:19:43.604856 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601417 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 18:19:43.605361 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601420 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 18:19:43.605361 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.601425 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 18:19:43.605361 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601541 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 18:19:43.605361 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601547 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 18:19:43.605361 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601550 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 18:19:43.605361 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601553 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 18:19:43.605361 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601556 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 18:19:43.605361 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601559 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 18:19:43.605361 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601562 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 18:19:43.605361 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601564 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 18:19:43.605361 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601567 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 18:19:43.605361 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601571 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 18:19:43.605361 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601575 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 18:19:43.605361 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601578 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 18:19:43.605361 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601582 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 18:19:43.605730 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601584 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 18:19:43.605730 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601587 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 18:19:43.605730 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601590 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 18:19:43.605730 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601592 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 18:19:43.605730 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601595 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 18:19:43.605730 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601598 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 18:19:43.605730 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601600 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 18:19:43.605730 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601604 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 18:19:43.605730 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601607 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 18:19:43.605730 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601611 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 18:19:43.605730 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601613 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 18:19:43.605730 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601616 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 18:19:43.605730 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601620 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 18:19:43.605730 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601622 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 18:19:43.605730 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601625 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 18:19:43.605730 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601627 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 18:19:43.605730 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601630 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 18:19:43.605730 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601634 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 18:19:43.605730 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601637 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 18:19:43.606329 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601640 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 18:19:43.606329 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601643 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 18:19:43.606329 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601646 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 18:19:43.606329 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601648 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 18:19:43.606329 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601650 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 18:19:43.606329 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601653 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 18:19:43.606329 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601655 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 18:19:43.606329 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601658 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 06 18:19:43.606329 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601660 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 18:19:43.606329 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601663 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 18:19:43.606329 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601665 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 18:19:43.606329 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601668 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 18:19:43.606329 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601670 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 18:19:43.606329 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601673 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 18:19:43.606329 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601675 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 18:19:43.606329 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601677 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 18:19:43.606329 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601680 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 18:19:43.606329 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601682 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 18:19:43.606329 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601685 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 18:19:43.606329 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601687 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 18:19:43.606789 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601690 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 18:19:43.606789 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601692 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 18:19:43.606789 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601695 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 18:19:43.606789 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601697 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 18:19:43.606789 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601699 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 18:19:43.606789 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601702 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 18:19:43.606789 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601704 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 18:19:43.606789 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601707 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 18:19:43.606789 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601709 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 18:19:43.606789 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601711 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 18:19:43.606789 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601714 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 18:19:43.606789 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601716 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 18:19:43.606789 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601719 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 18:19:43.606789 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601722 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 18:19:43.606789 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601724 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 18:19:43.606789 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601727 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 18:19:43.606789 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601729 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 18:19:43.606789 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601732 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 18:19:43.606789 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601734 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 18:19:43.606789 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601737 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 18:19:43.607293 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601739 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 18:19:43.607293 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601742 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 18:19:43.607293 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601744 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 18:19:43.607293 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601747 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 18:19:43.607293 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601750 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 18:19:43.607293 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601752 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 18:19:43.607293 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601755 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 18:19:43.607293 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601757 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 18:19:43.607293 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601759 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 18:19:43.607293 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601762 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 18:19:43.607293 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601764 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 18:19:43.607293 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601770 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 18:19:43.607293 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601772 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 18:19:43.607293 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601775 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 18:19:43.607293 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601777 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 18:19:43.607293 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601780 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 18:19:43.607293 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601782 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 18:19:43.607293 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601784 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 18:19:43.607293 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601787 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 18:19:43.607293 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601789 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 18:19:43.607770 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601792 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 18:19:43.607770 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601794 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 18:19:43.607770 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601797 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 18:19:43.607770 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601799 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 18:19:43.607770 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601802 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 18:19:43.607770 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601804 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 18:19:43.607770 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:43.601807 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 18:19:43.607770 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.601812 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 18:19:43.607770 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.601943 2566 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 18:19:43.607770 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.604391 2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 18:19:43.607770 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.605771 2566 server.go:1019] "Starting client certificate rotation" Jul 06 18:19:43.607770 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.605885 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 18:19:43.607770 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.606788 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 18:19:43.608748 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.608733 2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 18:19:43.609909 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.609889 2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 18:19:43.620004 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.619850 2566 log.go:25] "Validated CRI v1 runtime API" Jul 06 18:19:43.622598 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.622578 2566 log.go:25] "Validated CRI v1 image API" Jul 06 18:19:43.623496 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.623478 2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 18:19:43.627006 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.626987 2566 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 93d80a73-9b9a-43b3-8750-5464e60bd80a:/dev/nvme0n1p4 db15d5d5-dcbb-4c71-9e9a-7baaae85b133:/dev/nvme0n1p3] Jul 06 18:19:43.627068 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.627005 2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 18:19:43.632210 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.632097 2566 manager.go:216] Machine: {Timestamp:2026-07-06 18:19:43.630696956 +0000 UTC m=+0.357125906 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099171 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f6fdf22760273aa9ea16d212d1c48 SystemUUID:ec2f6fdf-2276-0273-aa9e-a16d212d1c48 BootID:41306a5e-bce0-471d-ad8d-77ce7b5e3c49 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:33:46:b6:bf:83 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:33:46:b6:bf:83 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:66:e8:bf:6b:0f:a5 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 18:19:43.632210 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.632207 2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 18:19:43.632350 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.632337 2566 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 18:19:43.635629 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.635475 2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 18:19:43.635769 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.635634 2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-98.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 18:19:43.637001 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.636986 2566 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 18:19:43.637001 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.637000 2566 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 18:19:43.637103 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.637007 2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 18:19:43.637616 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.637603 2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 18:19:43.638371 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.638354 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 18:19:43.638888 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.638862 2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 18:19:43.642205 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.642192 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 06 18:19:43.642337 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.642327 2566 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 18:19:43.642367 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.642361 2566 kubelet.go:486] "Attempting to sync node with API server" Jul 06 18:19:43.642398 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.642376 2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 18:19:43.642398 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.642392 2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 18:19:43.642451 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.642401 2566 kubelet.go:398] "Adding apiserver pod source" Jul 06 18:19:43.642451 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.642410 2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 18:19:43.642813 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.642789 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 18:19:43.642813 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.642809 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 18:19:43.646449 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.646424 2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 18:19:43.648768 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.648753 2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 18:19:43.648830 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.648784 2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 18:19:43.649881 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.649853 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 18:19:43.649881 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.649882 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 18:19:43.649956 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.649889 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 18:19:43.649956 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.649895 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 18:19:43.649956 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.649904 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 18:19:43.649956 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.649909 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 18:19:43.649956 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.649917 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 18:19:43.649956 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.649922 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 18:19:43.649956 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.649929 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 18:19:43.649956 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.649935 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 18:19:43.649956 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.649943 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 18:19:43.649956 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.649948 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 18:19:43.650797 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.650788 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 18:19:43.650797 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.650797 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 18:19:43.654079 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.654056 2566 server.go:1268] "Started kubelet" Jul 06 18:19:43.654202 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.654144 2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 18:19:43.654202 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.654156 2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 18:19:43.654313 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.654229 2566 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 18:19:43.654582 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:43.654549 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-98.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 18:19:43.654661 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:43.654647 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 18:19:43.654720 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.654662 2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-98.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 18:19:43.655091 ip-10-0-128-98 systemd[1]: Started Kubernetes Kubelet. Jul 06 18:19:43.660677 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.660511 2566 server.go:310] "Adding debug handlers to kubelet server" Jul 06 18:19:43.660677 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.660513 2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 18:19:43.662594 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:43.662572 2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 18:19:43.666028 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.666008 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vp762" Jul 06 18:19:43.666798 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.666774 2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 18:19:43.667042 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.666779 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 18:19:43.667042 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:43.666986 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-98.ec2.internal\" not found" Jul 06 18:19:43.667725 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.667596 2566 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 18:19:43.667725 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.667615 2566 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 18:19:43.667912 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.667752 2566 reconstruct.go:97] "Volume reconstruction finished" Jul 06 18:19:43.667912 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.667761 2566 reconciler.go:29] "Reconciler: start to sync state" Jul 06 18:19:43.667912 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.667776 2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 18:19:43.668114 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.668098 2566 factory.go:55] Registering systemd factory Jul 06 18:19:43.668190 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.668118 2566 factory.go:223] Registration of the systemd container factory successfully Jul 06 18:19:43.668190 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:43.666965 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-98.ec2.internal.18bfc64685a242c1 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-98.ec2.internal,UID:ip-10-0-128-98.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-98.ec2.internal,},FirstTimestamp:2026-07-06 18:19:43.654019777 +0000 UTC m=+0.380448725,LastTimestamp:2026-07-06 18:19:43.654019777 +0000 UTC m=+0.380448725,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-98.ec2.internal,}" Jul 06 18:19:43.668405 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.668377 2566 factory.go:153] Registering CRI-O factory Jul 06 18:19:43.668452 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.668410 2566 factory.go:223] Registration of the crio container factory successfully Jul 06 18:19:43.668492 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.668458 2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 18:19:43.668492 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.668477 2566 factory.go:103] Registering Raw factory Jul 06 18:19:43.668492 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.668487 2566 manager.go:1196] Started watching for new ooms in manager Jul 06 18:19:43.668926 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.668911 2566 manager.go:349] Starting recovery of all containers Jul 06 18:19:43.671495 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:43.671444 2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-98.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 18:19:43.671580 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:43.671560 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 18:19:43.675176 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.675146 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vp762" Jul 06 18:19:43.679855 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.679838 2566 manager.go:354] Recovery completed Jul 06 18:19:43.684207 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.684193 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 18:19:43.686169 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.686151 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-98.ec2.internal" event="NodeHasSufficientMemory" Jul 06 18:19:43.686250 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.686182 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-98.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 18:19:43.686250 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.686192 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-98.ec2.internal" event="NodeHasSufficientPID" Jul 06 18:19:43.686704 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.686687 2566 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 18:19:43.686777 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.686707 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 18:19:43.686777 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.686717 2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 18:19:43.686777 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.686734 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 06 18:19:43.687437 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.687418 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-98.ec2.internal" event="NodeHasSufficientMemory" Jul 06 18:19:43.687508 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.687454 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-98.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 18:19:43.687508 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.687471 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-98.ec2.internal" event="NodeHasSufficientPID" Jul 06 18:19:43.688133 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.688085 2566 policy_none.go:49] "None policy: Start" Jul 06 18:19:43.688235 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.688140 2566 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 18:19:43.688235 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.688156 2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 18:19:43.689054 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:43.688955 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-98.ec2.internal.18bfc646878ccde8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-98.ec2.internal,UID:ip-10-0-128-98.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-98.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-98.ec2.internal,},FirstTimestamp:2026-07-06 18:19:43.68616804 +0000 UTC m=+0.412596992,LastTimestamp:2026-07-06 18:19:43.68616804 +0000 UTC m=+0.412596992,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-98.ec2.internal,}" Jul 06 18:19:43.689525 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.689508 2566 policy_none.go:47] "Start" Jul 06 18:19:43.729489 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.729469 2566 manager.go:337] "Starting Device Plugin manager" Jul 06 18:19:43.729604 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:43.729566 2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 18:19:43.729604 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.729580 2566 server.go:87] "Starting device plugin registration server" Jul 06 18:19:43.729862 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.729849 2566 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 18:19:43.729934 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.729884 2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 18:19:43.729992 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.729975 2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 18:19:43.730066 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.730054 2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 18:19:43.730118 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.730067 2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 18:19:43.730625 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:43.730598 2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 18:19:43.730720 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:43.730655 2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-98.ec2.internal\" not found" Jul 06 18:19:43.830383 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.830352 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 18:19:43.831761 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.831745 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-98.ec2.internal" event="NodeHasSufficientMemory" Jul 06 18:19:43.831832 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.831783 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-98.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 18:19:43.831832 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.831796 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-98.ec2.internal" event="NodeHasSufficientPID" Jul 06 18:19:43.831832 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.831824 2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-98.ec2.internal" Jul 06 18:19:43.837251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.837203 2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-98.ec2.internal" Jul 06 18:19:43.837251 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:43.837228 2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-98.ec2.internal\": node \"ip-10-0-128-98.ec2.internal\" not found" Jul 06 18:19:43.841147 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.841110 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 18:19:43.842584 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.842558 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 18:19:43.842673 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.842597 2566 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 18:19:43.842673 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.842614 2566 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 18:19:43.842673 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:43.842646 2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 18:19:43.844251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.844234 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 18:19:43.852085 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:43.852065 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-98.ec2.internal\" not found" Jul 06 18:19:43.943606 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.943550 2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-98.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-98.ec2.internal"] Jul 06 18:19:43.943764 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.943671 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 18:19:43.945182 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.945161 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-98.ec2.internal" event="NodeHasSufficientMemory" Jul 06 18:19:43.945292 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.945194 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-98.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 18:19:43.945292 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.945216 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-98.ec2.internal" event="NodeHasSufficientPID" Jul 06 18:19:43.946502 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.946489 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 18:19:43.946637 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.946620 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-98.ec2.internal" Jul 06 18:19:43.946680 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.946658 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 18:19:43.947234 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.947214 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-98.ec2.internal" event="NodeHasSufficientMemory" Jul 06 18:19:43.947355 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.947241 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-98.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 18:19:43.947355 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.947256 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-98.ec2.internal" event="NodeHasSufficientPID" Jul 06 18:19:43.947355 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.947312 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-98.ec2.internal" event="NodeHasSufficientMemory" Jul 06 18:19:43.947355 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.947333 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-98.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 18:19:43.947355 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.947347 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-98.ec2.internal" event="NodeHasSufficientPID" Jul 06 18:19:43.948513 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.948499 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-98.ec2.internal" Jul 06 18:19:43.948567 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.948528 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 18:19:43.949284 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.949269 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-98.ec2.internal" event="NodeHasSufficientMemory" Jul 06 18:19:43.949434 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.949297 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-98.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 18:19:43.949434 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:43.949311 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-98.ec2.internal" event="NodeHasSufficientPID" Jul 06 18:19:43.952375 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:43.952358 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-98.ec2.internal\" not found" Jul 06 18:19:43.967445 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:43.967422 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-98.ec2.internal\" not found" node="ip-10-0-128-98.ec2.internal" Jul 06 18:19:43.971588 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:43.971572 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-98.ec2.internal\" not found" node="ip-10-0-128-98.ec2.internal" Jul 06 18:19:44.052681 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:44.052635 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-98.ec2.internal\" not found" Jul 06 18:19:44.070062 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.070038 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7ccbef026f2cc3be335a3fabd6f8311-config\") pod \"kube-apiserver-proxy-ip-10-0-128-98.ec2.internal\" (UID: \"b7ccbef026f2cc3be335a3fabd6f8311\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-98.ec2.internal" Jul 06 18:19:44.070143 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.070069 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ff73c77bfdbc9702b903107bb5e1e802-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-98.ec2.internal\" (UID: \"ff73c77bfdbc9702b903107bb5e1e802\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-98.ec2.internal" Jul 06 18:19:44.070143 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.070086 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff73c77bfdbc9702b903107bb5e1e802-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-98.ec2.internal\" (UID: \"ff73c77bfdbc9702b903107bb5e1e802\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-98.ec2.internal" Jul 06 18:19:44.153555 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:44.153478 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-98.ec2.internal\" not found" Jul 06 18:19:44.170974 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.170945 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ff73c77bfdbc9702b903107bb5e1e802-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-98.ec2.internal\" (UID: \"ff73c77bfdbc9702b903107bb5e1e802\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-98.ec2.internal" Jul 06 18:19:44.171046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.170977 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff73c77bfdbc9702b903107bb5e1e802-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-98.ec2.internal\" (UID: \"ff73c77bfdbc9702b903107bb5e1e802\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-98.ec2.internal" Jul 06 18:19:44.171046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.170995 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7ccbef026f2cc3be335a3fabd6f8311-config\") pod \"kube-apiserver-proxy-ip-10-0-128-98.ec2.internal\" (UID: \"b7ccbef026f2cc3be335a3fabd6f8311\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-98.ec2.internal" Jul 06 18:19:44.171151 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.171053 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ff73c77bfdbc9702b903107bb5e1e802-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-98.ec2.internal\" (UID: \"ff73c77bfdbc9702b903107bb5e1e802\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-98.ec2.internal" Jul 06 18:19:44.171151 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.171053 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7ccbef026f2cc3be335a3fabd6f8311-config\") pod \"kube-apiserver-proxy-ip-10-0-128-98.ec2.internal\" (UID: \"b7ccbef026f2cc3be335a3fabd6f8311\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-98.ec2.internal" Jul 06 18:19:44.171151 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.171099 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff73c77bfdbc9702b903107bb5e1e802-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-98.ec2.internal\" (UID: \"ff73c77bfdbc9702b903107bb5e1e802\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-98.ec2.internal" Jul 06 18:19:44.254152 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:44.254111 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-98.ec2.internal\" not found" Jul 06 18:19:44.271534 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.271508 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-98.ec2.internal" Jul 06 18:19:44.274447 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.274425 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-98.ec2.internal" Jul 06 18:19:44.354660 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:44.354607 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-98.ec2.internal\" not found" Jul 06 18:19:44.457981 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:44.457892 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-98.ec2.internal\" not found" Jul 06 18:19:44.483779 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.483747 2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 18:19:44.568047 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.568005 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-98.ec2.internal" Jul 06 18:19:44.578475 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.578449 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 18:19:44.580478 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.580462 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-98.ec2.internal" Jul 06 18:19:44.588528 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.588508 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 18:19:44.605101 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.605063 2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 18:19:44.605288 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.605258 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 18:19:44.605346 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.605258 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 18:19:44.642760 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.642723 2566 apiserver.go:52] "Watching apiserver" Jul 06 18:19:44.646262 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.646245 2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 18:19:44.650887 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.650865 2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 18:19:44.652309 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.652285 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-cz6db","openshift-network-diagnostics/network-check-target-n4j4l","openshift-network-operator/iptables-alerter-ldrkc","openshift-ovn-kubernetes/ovnkube-node-ffzjr","kube-system/konnectivity-agent-c2wvn","kube-system/kube-apiserver-proxy-ip-10-0-128-98.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqn7x","openshift-cluster-node-tuning-operator/tuned-kvr5q","openshift-dns/node-resolver-vttwc","openshift-image-registry/node-ca-xczs7","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-98.ec2.internal","openshift-multus/network-metrics-daemon-4gmr8","kube-system/global-pull-secret-syncer-5hbx6","openshift-multus/multus-additional-cni-plugins-r8tl5"] Jul 06 18:19:44.654195 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.654171 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqn7x" Jul 06 18:19:44.656657 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.656632 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:19:44.656808 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:44.656715 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5hbx6" podUID="3157c562-b33e-4c9c-b724-cdac9220bbb0" Jul 06 18:19:44.656966 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.656947 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 18:19:44.657419 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.657395 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 18:19:44.657419 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.657406 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 18:19:44.657570 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.657405 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-krr5r\"" Jul 06 18:19:44.658006 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.657985 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ldrkc" Jul 06 18:19:44.659437 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.659388 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.660750 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.660730 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c2wvn" Jul 06 18:19:44.660750 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.660745 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 18:19:44.660922 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.660782 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 18:19:44.661099 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.661084 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-n9jnl\"" Jul 06 18:19:44.661529 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.661511 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 18:19:44.662281 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.662253 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2r8cz\"" Jul 06 18:19:44.662409 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.662386 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-r8tl5" Jul 06 18:19:44.662839 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.662641 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 18:19:44.662839 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.662656 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 18:19:44.662839 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.662655 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 18:19:44.662839 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.662809 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 18:19:44.663758 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.663725 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vttwc" Jul 06 18:19:44.664020 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.664003 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 18:19:44.664307 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.664285 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 18:19:44.664406 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.664317 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 18:19:44.664511 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.664423 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 18:19:44.664578 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.664542 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8trdc\"" Jul 06 18:19:44.665207 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.665182 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 18:19:44.665686 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.665668 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vbtr4\"" Jul 06 18:19:44.665899 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.665875 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.666012 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.665969 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 18:19:44.666079 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.666013 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 18:19:44.666079 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.666016 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 18:19:44.666445 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.666423 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 18:19:44.666580 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.666564 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:19:44.666646 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:44.666628 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4j4l" podUID="f142192b-8eb7-419d-bfc0-a97171b652b7" Jul 06 18:19:44.666892 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.666869 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 18:19:44.667008 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.666901 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dc9vb\"" Jul 06 18:19:44.667008 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.666917 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 18:19:44.667008 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.666949 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 18:19:44.667965 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.667928 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xczs7" Jul 06 18:19:44.668715 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.668699 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 18:19:44.669021 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.669003 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jvj2d\"" Jul 06 18:19:44.669618 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.669601 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.670420 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.670404 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 18:19:44.670650 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.670635 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 18:19:44.670852 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.670838 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 18:19:44.670946 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.670928 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:19:44.671008 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:44.670987 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gmr8" podUID="f2c962e6-4904-4e1b-89f8-b6acb40f6a80" Jul 06 18:19:44.671256 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.671243 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cx6j6\"" Jul 06 18:19:44.672316 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.672300 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-v7rjh\"" Jul 06 18:19:44.672316 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.672289 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 18:19:44.672429 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.672325 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 18:19:44.672931 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.672915 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3157c562-b33e-4c9c-b724-cdac9220bbb0-dbus\") pod \"global-pull-secret-syncer-5hbx6\" (UID: \"3157c562-b33e-4c9c-b724-cdac9220bbb0\") " pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:19:44.672990 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.672957 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-sys\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.672990 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.672974 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c95fj\" (UniqueName: \"kubernetes.io/projected/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-kube-api-access-c95fj\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.673064 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673003 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/afc02035-3216-4c87-971b-d0cc3cb1006d-sys-fs\") pod \"aws-ebs-csi-driver-node-hqn7x\" (UID: \"afc02035-3216-4c87-971b-d0cc3cb1006d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqn7x" Jul 06 18:19:44.673117 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673064 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ee4e9134-c963-43ab-923a-42caaa210f43-iptables-alerter-script\") pod \"iptables-alerter-ldrkc\" (UID: \"ee4e9134-c963-43ab-923a-42caaa210f43\") " pod="openshift-network-operator/iptables-alerter-ldrkc" Jul 06 18:19:44.673117 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673088 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3157c562-b33e-4c9c-b724-cdac9220bbb0-kubelet-config\") pod \"global-pull-secret-syncer-5hbx6\" (UID: \"3157c562-b33e-4c9c-b724-cdac9220bbb0\") " pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:19:44.673226 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673121 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f-os-release\") pod \"multus-additional-cni-plugins-r8tl5\" (UID: \"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f\") " pod="openshift-multus/multus-additional-cni-plugins-r8tl5" Jul 06 18:19:44.673226 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673146 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvx8j\" (UniqueName: \"kubernetes.io/projected/f142192b-8eb7-419d-bfc0-a97171b652b7-kube-api-access-fvx8j\") pod \"network-check-target-n4j4l\" (UID: \"f142192b-8eb7-419d-bfc0-a97171b652b7\") " pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:19:44.673226 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673163 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-tmp\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.673226 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673178 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/afc02035-3216-4c87-971b-d0cc3cb1006d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hqn7x\" (UID: \"afc02035-3216-4c87-971b-d0cc3cb1006d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqn7x" Jul 06 18:19:44.673226 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673200 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1ede441d-79f2-4365-8a08-403bf043ef73-ovnkube-script-lib\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.673429 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673229 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-multus-socket-dir-parent\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.673429 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673255 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f-system-cni-dir\") pod \"multus-additional-cni-plugins-r8tl5\" (UID: \"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f\") " pod="openshift-multus/multus-additional-cni-plugins-r8tl5" Jul 06 18:19:44.673429 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673277 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-r8tl5\" (UID: \"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f\") " pod="openshift-multus/multus-additional-cni-plugins-r8tl5" Jul 06 18:19:44.673429 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673295 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-run\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.673429 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673312 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee4e9134-c963-43ab-923a-42caaa210f43-host-slash\") pod \"iptables-alerter-ldrkc\" (UID: \"ee4e9134-c963-43ab-923a-42caaa210f43\") " pod="openshift-network-operator/iptables-alerter-ldrkc" Jul 06 18:19:44.673429 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673327 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-host-kubelet\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.673429 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673358 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/991513c4-afc5-4f9a-b436-648417e2b0e4-agent-certs\") pod \"konnectivity-agent-c2wvn\" (UID: \"991513c4-afc5-4f9a-b436-648417e2b0e4\") " pod="kube-system/konnectivity-agent-c2wvn" Jul 06 18:19:44.673429 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673390 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-r8tl5\" (UID: \"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f\") " pod="openshift-multus/multus-additional-cni-plugins-r8tl5" Jul 06 18:19:44.673429 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673428 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/27d5cb4d-462b-42ab-a879-fc2b7ae008b5-serviceca\") pod \"node-ca-xczs7\" (UID: \"27d5cb4d-462b-42ab-a879-fc2b7ae008b5\") " pod="openshift-image-registry/node-ca-xczs7" Jul 06 18:19:44.673835 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673453 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpnmp\" (UniqueName: \"kubernetes.io/projected/27d5cb4d-462b-42ab-a879-fc2b7ae008b5-kube-api-access-xpnmp\") pod \"node-ca-xczs7\" (UID: \"27d5cb4d-462b-42ab-a879-fc2b7ae008b5\") " pod="openshift-image-registry/node-ca-xczs7" Jul 06 18:19:44.673835 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673476 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-systemd-units\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.673835 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673513 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-log-socket\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.673835 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673536 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1ede441d-79f2-4365-8a08-403bf043ef73-ovnkube-config\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.673835 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673560 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gl4s\" (UniqueName: \"kubernetes.io/projected/1ede441d-79f2-4365-8a08-403bf043ef73-kube-api-access-2gl4s\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.673835 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673593 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-host-var-lib-kubelet\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.673835 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673617 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-hostroot\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.673835 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673676 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-host-run-netns\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.673835 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673725 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-run-systemd\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.673835 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673757 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-host-run-k8s-cni-cncf-io\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.673835 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673784 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-etc-tuned\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.673835 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673810 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1ede441d-79f2-4365-8a08-403bf043ef73-ovn-node-metrics-cert\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.673835 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673834 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/29a00562-9fe6-473d-a4fd-31df8896cac8-cni-binary-copy\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.674345 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673857 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-multus-conf-dir\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.674345 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673919 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/27d5cb4d-462b-42ab-a879-fc2b7ae008b5-host\") pod \"node-ca-xczs7\" (UID: \"27d5cb4d-462b-42ab-a879-fc2b7ae008b5\") " pod="openshift-image-registry/node-ca-xczs7" Jul 06 18:19:44.674345 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673966 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-etc-sysconfig\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.674345 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.673986 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-host\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.674345 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674002 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a97d8c3-ea18-47d0-a315-59c693911c46-tmp-dir\") pod \"node-resolver-vttwc\" (UID: \"2a97d8c3-ea18-47d0-a315-59c693911c46\") " pod="openshift-dns/node-resolver-vttwc" Jul 06 18:19:44.674345 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674024 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-host-var-lib-cni-bin\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.674345 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674047 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-etc-kubernetes\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.674345 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674063 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-etc-sysctl-conf\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.674345 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674078 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-lib-modules\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.674345 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674093 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-host-run-multus-certs\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.674345 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674108 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-etc-kubernetes\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.674345 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674126 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5sz8\" (UniqueName: \"kubernetes.io/projected/adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f-kube-api-access-m5sz8\") pod \"multus-additional-cni-plugins-r8tl5\" (UID: \"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f\") " pod="openshift-multus/multus-additional-cni-plugins-r8tl5" Jul 06 18:19:44.674345 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674143 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-host-cni-netd\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.674345 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674160 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f-cnibin\") pod \"multus-additional-cni-plugins-r8tl5\" (UID: \"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f\") " pod="openshift-multus/multus-additional-cni-plugins-r8tl5" Jul 06 18:19:44.674345 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674175 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/afc02035-3216-4c87-971b-d0cc3cb1006d-etc-selinux\") pod \"aws-ebs-csi-driver-node-hqn7x\" (UID: \"afc02035-3216-4c87-971b-d0cc3cb1006d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqn7x" Jul 06 18:19:44.674345 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674196 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-host-slash\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.674345 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674225 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lznjl\" (UniqueName: \"kubernetes.io/projected/2a97d8c3-ea18-47d0-a315-59c693911c46-kube-api-access-lznjl\") pod \"node-resolver-vttwc\" (UID: \"2a97d8c3-ea18-47d0-a315-59c693911c46\") " pod="openshift-dns/node-resolver-vttwc" Jul 06 18:19:44.675012 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674259 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-multus-cni-dir\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.675012 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674285 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-r8tl5\" (UID: \"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f\") " pod="openshift-multus/multus-additional-cni-plugins-r8tl5" Jul 06 18:19:44.675012 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674303 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/afc02035-3216-4c87-971b-d0cc3cb1006d-socket-dir\") pod \"aws-ebs-csi-driver-node-hqn7x\" (UID: \"afc02035-3216-4c87-971b-d0cc3cb1006d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqn7x" Jul 06 18:19:44.675012 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674319 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-node-log\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.675012 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674334 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-host-run-netns\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.675012 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674349 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-host-var-lib-cni-multus\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.675012 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674371 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/29a00562-9fe6-473d-a4fd-31df8896cac8-multus-daemon-config\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.675012 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674385 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-etc-systemd\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.675012 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674400 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-host-cni-bin\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.675012 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674415 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1ede441d-79f2-4365-8a08-403bf043ef73-env-overrides\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.675012 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674431 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-cnibin\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.675012 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674447 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-os-release\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.675012 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674461 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f-cni-binary-copy\") pod \"multus-additional-cni-plugins-r8tl5\" (UID: \"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f\") " pod="openshift-multus/multus-additional-cni-plugins-r8tl5" Jul 06 18:19:44.675012 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674478 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-etc-modprobe-d\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.675012 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674492 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-var-lib-kubelet\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.675012 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674507 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw8tm\" (UniqueName: \"kubernetes.io/projected/ee4e9134-c963-43ab-923a-42caaa210f43-kube-api-access-pw8tm\") pod \"iptables-alerter-ldrkc\" (UID: \"ee4e9134-c963-43ab-923a-42caaa210f43\") " pod="openshift-network-operator/iptables-alerter-ldrkc" Jul 06 18:19:44.675474 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674541 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-etc-openvswitch\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.675474 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674572 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/afc02035-3216-4c87-971b-d0cc3cb1006d-registration-dir\") pod \"aws-ebs-csi-driver-node-hqn7x\" (UID: \"afc02035-3216-4c87-971b-d0cc3cb1006d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqn7x" Jul 06 18:19:44.675474 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674602 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2htj\" (UniqueName: \"kubernetes.io/projected/afc02035-3216-4c87-971b-d0cc3cb1006d-kube-api-access-r2htj\") pod \"aws-ebs-csi-driver-node-hqn7x\" (UID: \"afc02035-3216-4c87-971b-d0cc3cb1006d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqn7x" Jul 06 18:19:44.675474 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674652 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-var-lib-openvswitch\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.675474 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674680 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-run-openvswitch\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.675474 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674711 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.675474 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674741 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/991513c4-afc5-4f9a-b436-648417e2b0e4-konnectivity-ca\") pod \"konnectivity-agent-c2wvn\" (UID: \"991513c4-afc5-4f9a-b436-648417e2b0e4\") " pod="kube-system/konnectivity-agent-c2wvn" Jul 06 18:19:44.675474 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674763 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-etc-sysctl-d\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.675474 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674789 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/afc02035-3216-4c87-971b-d0cc3cb1006d-device-dir\") pod \"aws-ebs-csi-driver-node-hqn7x\" (UID: \"afc02035-3216-4c87-971b-d0cc3cb1006d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqn7x" Jul 06 18:19:44.675474 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674804 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-system-cni-dir\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.675474 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674819 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4mxs\" (UniqueName: \"kubernetes.io/projected/29a00562-9fe6-473d-a4fd-31df8896cac8-kube-api-access-l4mxs\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.675474 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674835 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3157c562-b33e-4c9c-b724-cdac9220bbb0-original-pull-secret\") pod \"global-pull-secret-syncer-5hbx6\" (UID: \"3157c562-b33e-4c9c-b724-cdac9220bbb0\") " pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:19:44.675474 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674857 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-run-ovn\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.675474 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674873 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-host-run-ovn-kubernetes\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.675474 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.674951 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2a97d8c3-ea18-47d0-a315-59c693911c46-hosts-file\") pod \"node-resolver-vttwc\" (UID: \"2a97d8c3-ea18-47d0-a315-59c693911c46\") " pod="openshift-dns/node-resolver-vttwc" Jul 06 18:19:44.677593 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.677547 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 18:14:44 +0000 UTC" deadline="2028-03-21 17:49:33.792401938 +0000 UTC" Jul 06 18:19:44.677662 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.677592 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14975h29m49.114812289s" Jul 06 18:19:44.682884 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.682862 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 18:19:44.704636 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.704616 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-q4k78" Jul 06 18:19:44.712142 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.712121 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-q4k78" Jul 06 18:19:44.768309 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.768281 2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 18:19:44.775188 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775154 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lznjl\" (UniqueName: \"kubernetes.io/projected/2a97d8c3-ea18-47d0-a315-59c693911c46-kube-api-access-lznjl\") pod \"node-resolver-vttwc\" (UID: \"2a97d8c3-ea18-47d0-a315-59c693911c46\") " pod="openshift-dns/node-resolver-vttwc" Jul 06 18:19:44.775314 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775199 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-multus-cni-dir\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.775314 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775232 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-r8tl5\" (UID: \"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f\") " pod="openshift-multus/multus-additional-cni-plugins-r8tl5" Jul 06 18:19:44.775434 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775356 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/afc02035-3216-4c87-971b-d0cc3cb1006d-socket-dir\") pod \"aws-ebs-csi-driver-node-hqn7x\" (UID: \"afc02035-3216-4c87-971b-d0cc3cb1006d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqn7x" Jul 06 18:19:44.775434 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775365 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-multus-cni-dir\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.775434 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775385 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-node-log\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.775434 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775410 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-host-run-netns\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.775639 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775433 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-host-var-lib-cni-multus\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.775639 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775458 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/29a00562-9fe6-473d-a4fd-31df8896cac8-multus-daemon-config\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.775639 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775487 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-etc-systemd\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.775639 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775532 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/afc02035-3216-4c87-971b-d0cc3cb1006d-socket-dir\") pod \"aws-ebs-csi-driver-node-hqn7x\" (UID: \"afc02035-3216-4c87-971b-d0cc3cb1006d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqn7x" Jul 06 18:19:44.775639 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775565 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-host-var-lib-cni-multus\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.775639 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775590 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-host-cni-bin\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.775639 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775541 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-host-cni-bin\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.775639 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775620 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-node-log\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.775639 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775633 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1ede441d-79f2-4365-8a08-403bf043ef73-env-overrides\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.776132 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775660 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-cnibin\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.776132 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775667 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-host-run-netns\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.776132 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775685 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-os-release\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.776132 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775710 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f-cni-binary-copy\") pod \"multus-additional-cni-plugins-r8tl5\" (UID: \"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f\") " pod="openshift-multus/multus-additional-cni-plugins-r8tl5" Jul 06 18:19:44.776132 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775736 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-etc-modprobe-d\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.776132 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775761 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-var-lib-kubelet\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.776132 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775786 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw8tm\" (UniqueName: \"kubernetes.io/projected/ee4e9134-c963-43ab-923a-42caaa210f43-kube-api-access-pw8tm\") pod \"iptables-alerter-ldrkc\" (UID: \"ee4e9134-c963-43ab-923a-42caaa210f43\") " pod="openshift-network-operator/iptables-alerter-ldrkc" Jul 06 18:19:44.776132 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775812 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-etc-openvswitch\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.776132 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775836 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/afc02035-3216-4c87-971b-d0cc3cb1006d-registration-dir\") pod \"aws-ebs-csi-driver-node-hqn7x\" (UID: \"afc02035-3216-4c87-971b-d0cc3cb1006d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqn7x" Jul 06 18:19:44.776132 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775864 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2htj\" (UniqueName: \"kubernetes.io/projected/afc02035-3216-4c87-971b-d0cc3cb1006d-kube-api-access-r2htj\") pod \"aws-ebs-csi-driver-node-hqn7x\" (UID: \"afc02035-3216-4c87-971b-d0cc3cb1006d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqn7x" Jul 06 18:19:44.776132 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775890 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-var-lib-openvswitch\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.776132 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775935 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-run-openvswitch\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.776132 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775961 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.776132 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775988 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-r8tl5\" (UID: \"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f\") " pod="openshift-multus/multus-additional-cni-plugins-r8tl5" Jul 06 18:19:44.776132 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.775991 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/991513c4-afc5-4f9a-b436-648417e2b0e4-konnectivity-ca\") pod \"konnectivity-agent-c2wvn\" (UID: \"991513c4-afc5-4f9a-b436-648417e2b0e4\") " pod="kube-system/konnectivity-agent-c2wvn" Jul 06 18:19:44.776132 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776068 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-etc-sysctl-d\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.776132 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776091 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/afc02035-3216-4c87-971b-d0cc3cb1006d-device-dir\") pod \"aws-ebs-csi-driver-node-hqn7x\" (UID: \"afc02035-3216-4c87-971b-d0cc3cb1006d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqn7x" Jul 06 18:19:44.777111 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776109 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-system-cni-dir\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.777111 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776125 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4mxs\" (UniqueName: \"kubernetes.io/projected/29a00562-9fe6-473d-a4fd-31df8896cac8-kube-api-access-l4mxs\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.777111 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776142 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3157c562-b33e-4c9c-b724-cdac9220bbb0-original-pull-secret\") pod \"global-pull-secret-syncer-5hbx6\" (UID: \"3157c562-b33e-4c9c-b724-cdac9220bbb0\") " pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:19:44.777111 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776157 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-run-ovn\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.777111 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776174 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-host-run-ovn-kubernetes\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.777111 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776190 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2a97d8c3-ea18-47d0-a315-59c693911c46-hosts-file\") pod \"node-resolver-vttwc\" (UID: \"2a97d8c3-ea18-47d0-a315-59c693911c46\") " pod="openshift-dns/node-resolver-vttwc" Jul 06 18:19:44.777111 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776203 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-system-cni-dir\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.777111 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776210 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2c962e6-4904-4e1b-89f8-b6acb40f6a80-metrics-certs\") pod \"network-metrics-daemon-4gmr8\" (UID: \"f2c962e6-4904-4e1b-89f8-b6acb40f6a80\") " pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:19:44.777111 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776236 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-etc-systemd\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.777111 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776242 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3157c562-b33e-4c9c-b724-cdac9220bbb0-dbus\") pod \"global-pull-secret-syncer-5hbx6\" (UID: \"3157c562-b33e-4c9c-b724-cdac9220bbb0\") " pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:19:44.777111 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776266 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-sys\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.777111 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776308 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-sys\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.777111 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776432 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-etc-sysctl-d\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.777111 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:44.776455 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 18:19:44.777111 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776485 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/afc02035-3216-4c87-971b-d0cc3cb1006d-device-dir\") pod \"aws-ebs-csi-driver-node-hqn7x\" (UID: \"afc02035-3216-4c87-971b-d0cc3cb1006d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqn7x" Jul 06 18:19:44.777111 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:44.776513 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3157c562-b33e-4c9c-b724-cdac9220bbb0-original-pull-secret podName:3157c562-b33e-4c9c-b724-cdac9220bbb0 nodeName:}" failed. No retries permitted until 2026-07-06 18:19:45.276498391 +0000 UTC m=+2.002927329 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3157c562-b33e-4c9c-b724-cdac9220bbb0-original-pull-secret") pod "global-pull-secret-syncer-5hbx6" (UID: "3157c562-b33e-4c9c-b724-cdac9220bbb0") : object "kube-system"/"original-pull-secret" not registered Jul 06 18:19:44.777111 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776558 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-cnibin\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.778076 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776573 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-var-lib-openvswitch\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.778076 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776581 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3157c562-b33e-4c9c-b724-cdac9220bbb0-dbus\") pod \"global-pull-secret-syncer-5hbx6\" (UID: \"3157c562-b33e-4c9c-b724-cdac9220bbb0\") " pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:19:44.778076 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776558 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-run-ovn\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.778076 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776569 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/afc02035-3216-4c87-971b-d0cc3cb1006d-registration-dir\") pod \"aws-ebs-csi-driver-node-hqn7x\" (UID: \"afc02035-3216-4c87-971b-d0cc3cb1006d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqn7x" Jul 06 18:19:44.778076 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776599 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-run-openvswitch\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.778076 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776184 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/29a00562-9fe6-473d-a4fd-31df8896cac8-multus-daemon-config\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.778076 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776595 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-host-run-ovn-kubernetes\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.778076 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776507 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/991513c4-afc5-4f9a-b436-648417e2b0e4-konnectivity-ca\") pod \"konnectivity-agent-c2wvn\" (UID: \"991513c4-afc5-4f9a-b436-648417e2b0e4\") " pod="kube-system/konnectivity-agent-c2wvn" Jul 06 18:19:44.778076 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776155 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1ede441d-79f2-4365-8a08-403bf043ef73-env-overrides\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.778076 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776649 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.778076 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776654 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-os-release\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.778076 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776686 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-etc-openvswitch\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.778076 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776727 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c95fj\" (UniqueName: \"kubernetes.io/projected/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-kube-api-access-c95fj\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.778076 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776732 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-etc-modprobe-d\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.778076 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776758 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/afc02035-3216-4c87-971b-d0cc3cb1006d-sys-fs\") pod \"aws-ebs-csi-driver-node-hqn7x\" (UID: \"afc02035-3216-4c87-971b-d0cc3cb1006d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqn7x" Jul 06 18:19:44.778076 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776797 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-var-lib-kubelet\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.778076 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776825 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2a97d8c3-ea18-47d0-a315-59c693911c46-hosts-file\") pod \"node-resolver-vttwc\" (UID: \"2a97d8c3-ea18-47d0-a315-59c693911c46\") " pod="openshift-dns/node-resolver-vttwc" Jul 06 18:19:44.778732 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776839 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ee4e9134-c963-43ab-923a-42caaa210f43-iptables-alerter-script\") pod \"iptables-alerter-ldrkc\" (UID: \"ee4e9134-c963-43ab-923a-42caaa210f43\") " pod="openshift-network-operator/iptables-alerter-ldrkc" Jul 06 18:19:44.778732 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776849 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/afc02035-3216-4c87-971b-d0cc3cb1006d-sys-fs\") pod \"aws-ebs-csi-driver-node-hqn7x\" (UID: \"afc02035-3216-4c87-971b-d0cc3cb1006d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqn7x" Jul 06 18:19:44.778732 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776894 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3157c562-b33e-4c9c-b724-cdac9220bbb0-kubelet-config\") pod \"global-pull-secret-syncer-5hbx6\" (UID: \"3157c562-b33e-4c9c-b724-cdac9220bbb0\") " pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:19:44.778732 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.776995 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f-os-release\") pod \"multus-additional-cni-plugins-r8tl5\" (UID: \"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f\") " pod="openshift-multus/multus-additional-cni-plugins-r8tl5" Jul 06 18:19:44.778732 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777003 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3157c562-b33e-4c9c-b724-cdac9220bbb0-kubelet-config\") pod \"global-pull-secret-syncer-5hbx6\" (UID: \"3157c562-b33e-4c9c-b724-cdac9220bbb0\") " pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:19:44.778732 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777041 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvx8j\" (UniqueName: \"kubernetes.io/projected/f142192b-8eb7-419d-bfc0-a97171b652b7-kube-api-access-fvx8j\") pod \"network-check-target-n4j4l\" (UID: \"f142192b-8eb7-419d-bfc0-a97171b652b7\") " pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:19:44.778732 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777055 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f-os-release\") pod \"multus-additional-cni-plugins-r8tl5\" (UID: \"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f\") " pod="openshift-multus/multus-additional-cni-plugins-r8tl5" Jul 06 18:19:44.778732 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777086 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-tmp\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.778732 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777208 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f-cni-binary-copy\") pod \"multus-additional-cni-plugins-r8tl5\" (UID: \"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f\") " pod="openshift-multus/multus-additional-cni-plugins-r8tl5" Jul 06 18:19:44.778732 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777204 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/afc02035-3216-4c87-971b-d0cc3cb1006d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hqn7x\" (UID: \"afc02035-3216-4c87-971b-d0cc3cb1006d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqn7x" Jul 06 18:19:44.778732 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777278 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/afc02035-3216-4c87-971b-d0cc3cb1006d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hqn7x\" (UID: \"afc02035-3216-4c87-971b-d0cc3cb1006d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqn7x" Jul 06 18:19:44.778732 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777278 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1ede441d-79f2-4365-8a08-403bf043ef73-ovnkube-script-lib\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.778732 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777315 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-multus-socket-dir-parent\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.778732 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777337 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f-system-cni-dir\") pod \"multus-additional-cni-plugins-r8tl5\" (UID: \"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f\") " pod="openshift-multus/multus-additional-cni-plugins-r8tl5" Jul 06 18:19:44.778732 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777360 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-r8tl5\" (UID: \"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f\") " pod="openshift-multus/multus-additional-cni-plugins-r8tl5" Jul 06 18:19:44.778732 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777386 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-run\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.778732 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777397 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-multus-socket-dir-parent\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.779840 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777406 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ee4e9134-c963-43ab-923a-42caaa210f43-iptables-alerter-script\") pod \"iptables-alerter-ldrkc\" (UID: \"ee4e9134-c963-43ab-923a-42caaa210f43\") " pod="openshift-network-operator/iptables-alerter-ldrkc" Jul 06 18:19:44.779840 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777417 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee4e9134-c963-43ab-923a-42caaa210f43-host-slash\") pod \"iptables-alerter-ldrkc\" (UID: \"ee4e9134-c963-43ab-923a-42caaa210f43\") " pod="openshift-network-operator/iptables-alerter-ldrkc" Jul 06 18:19:44.779840 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777408 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f-system-cni-dir\") pod \"multus-additional-cni-plugins-r8tl5\" (UID: \"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f\") " pod="openshift-multus/multus-additional-cni-plugins-r8tl5" Jul 06 18:19:44.779840 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777446 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-host-kubelet\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.779840 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777465 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/991513c4-afc5-4f9a-b436-648417e2b0e4-agent-certs\") pod \"konnectivity-agent-c2wvn\" (UID: \"991513c4-afc5-4f9a-b436-648417e2b0e4\") " pod="kube-system/konnectivity-agent-c2wvn" Jul 06 18:19:44.779840 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777487 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-run\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.779840 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777526 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-host-kubelet\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.779840 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777521 2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 18:19:44.779840 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777526 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-r8tl5\" (UID: \"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f\") " pod="openshift-multus/multus-additional-cni-plugins-r8tl5" Jul 06 18:19:44.779840 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777573 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee4e9134-c963-43ab-923a-42caaa210f43-host-slash\") pod \"iptables-alerter-ldrkc\" (UID: \"ee4e9134-c963-43ab-923a-42caaa210f43\") " pod="openshift-network-operator/iptables-alerter-ldrkc" Jul 06 18:19:44.779840 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777669 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/27d5cb4d-462b-42ab-a879-fc2b7ae008b5-serviceca\") pod \"node-ca-xczs7\" (UID: \"27d5cb4d-462b-42ab-a879-fc2b7ae008b5\") " pod="openshift-image-registry/node-ca-xczs7" Jul 06 18:19:44.779840 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777729 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpnmp\" (UniqueName: \"kubernetes.io/projected/27d5cb4d-462b-42ab-a879-fc2b7ae008b5-kube-api-access-xpnmp\") pod \"node-ca-xczs7\" (UID: \"27d5cb4d-462b-42ab-a879-fc2b7ae008b5\") " pod="openshift-image-registry/node-ca-xczs7" Jul 06 18:19:44.779840 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777754 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-systemd-units\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.779840 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777781 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1ede441d-79f2-4365-8a08-403bf043ef73-ovnkube-script-lib\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.779840 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777870 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-systemd-units\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.779840 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777947 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-log-socket\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.779840 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.778021 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1ede441d-79f2-4365-8a08-403bf043ef73-ovnkube-config\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.779840 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.777986 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-log-socket\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.780849 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.778050 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gl4s\" (UniqueName: \"kubernetes.io/projected/1ede441d-79f2-4365-8a08-403bf043ef73-kube-api-access-2gl4s\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.780849 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.778077 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-host-var-lib-kubelet\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.780849 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.778102 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-hostroot\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.780849 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.778111 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-r8tl5\" (UID: \"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f\") " pod="openshift-multus/multus-additional-cni-plugins-r8tl5" Jul 06 18:19:44.780849 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.778128 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-host-run-netns\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.780849 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.778144 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-r8tl5\" (UID: \"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f\") " pod="openshift-multus/multus-additional-cni-plugins-r8tl5" Jul 06 18:19:44.780849 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.778185 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-host-var-lib-kubelet\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.780849 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.778189 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-hostroot\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.780849 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.778238 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-host-run-netns\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.780849 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.778276 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-run-systemd\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.780849 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.778309 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-host-run-k8s-cni-cncf-io\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.780849 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.778348 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-etc-tuned\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.780849 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.778370 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1ede441d-79f2-4365-8a08-403bf043ef73-ovn-node-metrics-cert\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.780849 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.778381 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/27d5cb4d-462b-42ab-a879-fc2b7ae008b5-serviceca\") pod \"node-ca-xczs7\" (UID: \"27d5cb4d-462b-42ab-a879-fc2b7ae008b5\") " pod="openshift-image-registry/node-ca-xczs7" Jul 06 18:19:44.780849 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.778394 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/29a00562-9fe6-473d-a4fd-31df8896cac8-cni-binary-copy\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.780849 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.778414 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-run-systemd\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.780849 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.778431 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-multus-conf-dir\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.780849 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.778484 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-multus-conf-dir\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.781726 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.778728 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1ede441d-79f2-4365-8a08-403bf043ef73-ovnkube-config\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.781726 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.778814 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-host-run-k8s-cni-cncf-io\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.781726 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.778980 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/29a00562-9fe6-473d-a4fd-31df8896cac8-cni-binary-copy\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.781726 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.778999 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/27d5cb4d-462b-42ab-a879-fc2b7ae008b5-host\") pod \"node-ca-xczs7\" (UID: \"27d5cb4d-462b-42ab-a879-fc2b7ae008b5\") " pod="openshift-image-registry/node-ca-xczs7" Jul 06 18:19:44.781726 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.779083 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-etc-sysconfig\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.781726 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.779130 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-host\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.781726 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.779132 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/27d5cb4d-462b-42ab-a879-fc2b7ae008b5-host\") pod \"node-ca-xczs7\" (UID: \"27d5cb4d-462b-42ab-a879-fc2b7ae008b5\") " pod="openshift-image-registry/node-ca-xczs7" Jul 06 18:19:44.781726 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.779155 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a97d8c3-ea18-47d0-a315-59c693911c46-tmp-dir\") pod \"node-resolver-vttwc\" (UID: \"2a97d8c3-ea18-47d0-a315-59c693911c46\") " pod="openshift-dns/node-resolver-vttwc" Jul 06 18:19:44.781726 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.779183 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-host-var-lib-cni-bin\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.781726 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.779213 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-etc-kubernetes\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.781726 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.779227 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-host\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.781726 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.779239 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-etc-sysctl-conf\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.781726 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.779263 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-lib-modules\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.781726 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.779269 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-host-var-lib-cni-bin\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.781726 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.779294 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmc52\" (UniqueName: \"kubernetes.io/projected/f2c962e6-4904-4e1b-89f8-b6acb40f6a80-kube-api-access-gmc52\") pod \"network-metrics-daemon-4gmr8\" (UID: \"f2c962e6-4904-4e1b-89f8-b6acb40f6a80\") " pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:19:44.781726 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.779323 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-host-run-multus-certs\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.781726 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.779346 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-etc-kubernetes\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.781726 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.779375 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5sz8\" (UniqueName: \"kubernetes.io/projected/adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f-kube-api-access-m5sz8\") pod \"multus-additional-cni-plugins-r8tl5\" (UID: \"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f\") " pod="openshift-multus/multus-additional-cni-plugins-r8tl5" Jul 06 18:19:44.782328 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.779407 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-host-cni-netd\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.782328 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.779434 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f-cnibin\") pod \"multus-additional-cni-plugins-r8tl5\" (UID: \"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f\") " pod="openshift-multus/multus-additional-cni-plugins-r8tl5" Jul 06 18:19:44.782328 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.779459 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/afc02035-3216-4c87-971b-d0cc3cb1006d-etc-selinux\") pod \"aws-ebs-csi-driver-node-hqn7x\" (UID: \"afc02035-3216-4c87-971b-d0cc3cb1006d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqn7x" Jul 06 18:19:44.782328 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.779484 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-host-slash\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.782328 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.779488 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-etc-kubernetes\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.782328 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.779526 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a97d8c3-ea18-47d0-a315-59c693911c46-tmp-dir\") pod \"node-resolver-vttwc\" (UID: \"2a97d8c3-ea18-47d0-a315-59c693911c46\") " pod="openshift-dns/node-resolver-vttwc" Jul 06 18:19:44.782328 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.779588 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-host-slash\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.782328 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.779568 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-etc-kubernetes\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.782328 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.779633 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f-cnibin\") pod \"multus-additional-cni-plugins-r8tl5\" (UID: \"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f\") " pod="openshift-multus/multus-additional-cni-plugins-r8tl5" Jul 06 18:19:44.782328 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.779187 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-etc-sysconfig\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.782328 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.779691 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/afc02035-3216-4c87-971b-d0cc3cb1006d-etc-selinux\") pod \"aws-ebs-csi-driver-node-hqn7x\" (UID: \"afc02035-3216-4c87-971b-d0cc3cb1006d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqn7x" Jul 06 18:19:44.782328 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.779746 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1ede441d-79f2-4365-8a08-403bf043ef73-host-cni-netd\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.782328 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.779779 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/29a00562-9fe6-473d-a4fd-31df8896cac8-host-run-multus-certs\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.782328 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.779830 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-lib-modules\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.782328 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.779895 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-etc-sysctl-conf\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.782328 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.781336 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/991513c4-afc5-4f9a-b436-648417e2b0e4-agent-certs\") pod \"konnectivity-agent-c2wvn\" (UID: \"991513c4-afc5-4f9a-b436-648417e2b0e4\") " pod="kube-system/konnectivity-agent-c2wvn" Jul 06 18:19:44.782328 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.781574 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1ede441d-79f2-4365-8a08-403bf043ef73-ovn-node-metrics-cert\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.782328 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.781838 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-etc-tuned\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.782849 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.782019 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-tmp\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.785641 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:44.785622 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 18:19:44.785790 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:44.785649 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 18:19:44.785790 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:44.785662 2566 projected.go:196] Error preparing data for projected volume kube-api-access-fvx8j for pod openshift-network-diagnostics/network-check-target-n4j4l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:19:44.785790 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:44.785732 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f142192b-8eb7-419d-bfc0-a97171b652b7-kube-api-access-fvx8j podName:f142192b-8eb7-419d-bfc0-a97171b652b7 nodeName:}" failed. No retries permitted until 2026-07-06 18:19:45.285713938 +0000 UTC m=+2.012142879 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fvx8j" (UniqueName: "kubernetes.io/projected/f142192b-8eb7-419d-bfc0-a97171b652b7-kube-api-access-fvx8j") pod "network-check-target-n4j4l" (UID: "f142192b-8eb7-419d-bfc0-a97171b652b7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:19:44.788405 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.788132 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2htj\" (UniqueName: \"kubernetes.io/projected/afc02035-3216-4c87-971b-d0cc3cb1006d-kube-api-access-r2htj\") pod \"aws-ebs-csi-driver-node-hqn7x\" (UID: \"afc02035-3216-4c87-971b-d0cc3cb1006d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqn7x" Jul 06 18:19:44.788405 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.788356 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpnmp\" (UniqueName: \"kubernetes.io/projected/27d5cb4d-462b-42ab-a879-fc2b7ae008b5-kube-api-access-xpnmp\") pod \"node-ca-xczs7\" (UID: \"27d5cb4d-462b-42ab-a879-fc2b7ae008b5\") " pod="openshift-image-registry/node-ca-xczs7" Jul 06 18:19:44.788669 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.788641 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw8tm\" (UniqueName: \"kubernetes.io/projected/ee4e9134-c963-43ab-923a-42caaa210f43-kube-api-access-pw8tm\") pod \"iptables-alerter-ldrkc\" (UID: \"ee4e9134-c963-43ab-923a-42caaa210f43\") " pod="openshift-network-operator/iptables-alerter-ldrkc" Jul 06 18:19:44.789142 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.789116 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c95fj\" (UniqueName: \"kubernetes.io/projected/c61cfa3e-6fda-4abf-8245-baeb901eb1f8-kube-api-access-c95fj\") pod \"tuned-kvr5q\" (UID: \"c61cfa3e-6fda-4abf-8245-baeb901eb1f8\") " pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.789373 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.789342 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lznjl\" (UniqueName: \"kubernetes.io/projected/2a97d8c3-ea18-47d0-a315-59c693911c46-kube-api-access-lznjl\") pod \"node-resolver-vttwc\" (UID: \"2a97d8c3-ea18-47d0-a315-59c693911c46\") " pod="openshift-dns/node-resolver-vttwc" Jul 06 18:19:44.789658 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.789638 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4mxs\" (UniqueName: \"kubernetes.io/projected/29a00562-9fe6-473d-a4fd-31df8896cac8-kube-api-access-l4mxs\") pod \"multus-cz6db\" (UID: \"29a00562-9fe6-473d-a4fd-31df8896cac8\") " pod="openshift-multus/multus-cz6db" Jul 06 18:19:44.790187 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.790165 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gl4s\" (UniqueName: \"kubernetes.io/projected/1ede441d-79f2-4365-8a08-403bf043ef73-kube-api-access-2gl4s\") pod \"ovnkube-node-ffzjr\" (UID: \"1ede441d-79f2-4365-8a08-403bf043ef73\") " pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:44.792069 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.792053 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5sz8\" (UniqueName: \"kubernetes.io/projected/adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f-kube-api-access-m5sz8\") pod \"multus-additional-cni-plugins-r8tl5\" (UID: \"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f\") " pod="openshift-multus/multus-additional-cni-plugins-r8tl5" Jul 06 18:19:44.806859 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.806836 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xczs7" Jul 06 18:19:44.813817 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.813797 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" Jul 06 18:19:44.880401 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.880367 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmc52\" (UniqueName: \"kubernetes.io/projected/f2c962e6-4904-4e1b-89f8-b6acb40f6a80-kube-api-access-gmc52\") pod \"network-metrics-daemon-4gmr8\" (UID: \"f2c962e6-4904-4e1b-89f8-b6acb40f6a80\") " pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:19:44.880574 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.880550 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2c962e6-4904-4e1b-89f8-b6acb40f6a80-metrics-certs\") pod \"network-metrics-daemon-4gmr8\" (UID: \"f2c962e6-4904-4e1b-89f8-b6acb40f6a80\") " pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:19:44.880729 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:44.880706 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:19:44.880798 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:44.880786 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2c962e6-4904-4e1b-89f8-b6acb40f6a80-metrics-certs podName:f2c962e6-4904-4e1b-89f8-b6acb40f6a80 nodeName:}" failed. No retries permitted until 2026-07-06 18:19:45.380764403 +0000 UTC m=+2.107193353 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f2c962e6-4904-4e1b-89f8-b6acb40f6a80-metrics-certs") pod "network-metrics-daemon-4gmr8" (UID: "f2c962e6-4904-4e1b-89f8-b6acb40f6a80") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:19:44.886040 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:44.885999 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff73c77bfdbc9702b903107bb5e1e802.slice/crio-3662f980179a9e3e5f952b8636afce3971c34349e723d245a7b7132b72ed9c44 WatchSource:0}: Error finding container 3662f980179a9e3e5f952b8636afce3971c34349e723d245a7b7132b72ed9c44: Status 404 returned error can't find the container with id 3662f980179a9e3e5f952b8636afce3971c34349e723d245a7b7132b72ed9c44 Jul 06 18:19:44.886629 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:44.886609 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7ccbef026f2cc3be335a3fabd6f8311.slice/crio-074b8e53ef0f3f58a9c4e8dda0eb01c1713ec97bfa4d24960e3cafec39f2f09d WatchSource:0}: Error finding container 074b8e53ef0f3f58a9c4e8dda0eb01c1713ec97bfa4d24960e3cafec39f2f09d: Status 404 returned error can't find the container with id 074b8e53ef0f3f58a9c4e8dda0eb01c1713ec97bfa4d24960e3cafec39f2f09d Jul 06 18:19:44.890895 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.890838 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 18:19:44.892976 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.892934 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmc52\" (UniqueName: \"kubernetes.io/projected/f2c962e6-4904-4e1b-89f8-b6acb40f6a80-kube-api-access-gmc52\") pod \"network-metrics-daemon-4gmr8\" (UID: \"f2c962e6-4904-4e1b-89f8-b6acb40f6a80\") " pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:19:44.981997 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.981869 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqn7x" Jul 06 18:19:44.991358 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:44.991329 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafc02035_3216_4c87_971b_d0cc3cb1006d.slice/crio-9690159a1997e05e5b8c30b7acc5dfb0df6d000ad0a5d731fa1bce84ad47f1ff WatchSource:0}: Error finding container 9690159a1997e05e5b8c30b7acc5dfb0df6d000ad0a5d731fa1bce84ad47f1ff: Status 404 returned error can't find the container with id 9690159a1997e05e5b8c30b7acc5dfb0df6d000ad0a5d731fa1bce84ad47f1ff Jul 06 18:19:44.997735 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:44.997710 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ldrkc" Jul 06 18:19:45.001633 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:45.001617 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 18:19:45.006668 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:45.006644 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee4e9134_c963_43ab_923a_42caaa210f43.slice/crio-06531132804b51e8b02c699914361b402c2e96b9e470317104d9e31da7e6bdc6 WatchSource:0}: Error finding container 06531132804b51e8b02c699914361b402c2e96b9e470317104d9e31da7e6bdc6: Status 404 returned error can't find the container with id 06531132804b51e8b02c699914361b402c2e96b9e470317104d9e31da7e6bdc6 Jul 06 18:19:45.008116 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:45.008099 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:19:45.012588 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:45.012561 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c2wvn" Jul 06 18:19:45.016311 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:45.016290 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ede441d_79f2_4365_8a08_403bf043ef73.slice/crio-d9148c8cf12caff9dfafb26bcef475a43e63ea1379afa700910c403534ce4b09 WatchSource:0}: Error finding container d9148c8cf12caff9dfafb26bcef475a43e63ea1379afa700910c403534ce4b09: Status 404 returned error can't find the container with id d9148c8cf12caff9dfafb26bcef475a43e63ea1379afa700910c403534ce4b09 Jul 06 18:19:45.021370 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:45.021349 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod991513c4_afc5_4f9a_b436_648417e2b0e4.slice/crio-c22067df9e964ed2a8bff8cf33c4383c13ac48167523bac64689a48b4eb2da36 WatchSource:0}: Error finding container c22067df9e964ed2a8bff8cf33c4383c13ac48167523bac64689a48b4eb2da36: Status 404 returned error can't find the container with id c22067df9e964ed2a8bff8cf33c4383c13ac48167523bac64689a48b4eb2da36 Jul 06 18:19:45.029267 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:45.029247 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-r8tl5" Jul 06 18:19:45.037721 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:45.037697 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadc427cd_3561_4ff2_b61a_fe4fbe4cdf7f.slice/crio-cc1672a988ccb67ccca3553ec17e1dc470eff5b23ed295d5182d2432640ac1fd WatchSource:0}: Error finding container cc1672a988ccb67ccca3553ec17e1dc470eff5b23ed295d5182d2432640ac1fd: Status 404 returned error can't find the container with id cc1672a988ccb67ccca3553ec17e1dc470eff5b23ed295d5182d2432640ac1fd Jul 06 18:19:45.047007 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:45.046991 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vttwc" Jul 06 18:19:45.054592 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:45.054567 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a97d8c3_ea18_47d0_a315_59c693911c46.slice/crio-a31be7049697abf51f69d964db3c570da4e4ba38babcfba3578aa5fe42fc561b WatchSource:0}: Error finding container a31be7049697abf51f69d964db3c570da4e4ba38babcfba3578aa5fe42fc561b: Status 404 returned error can't find the container with id a31be7049697abf51f69d964db3c570da4e4ba38babcfba3578aa5fe42fc561b Jul 06 18:19:45.071719 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:45.071697 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cz6db" Jul 06 18:19:45.079440 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:45.079417 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29a00562_9fe6_473d_a4fd_31df8896cac8.slice/crio-4425a59bfdfbaa52f2a12b25829ab9be2b8aae13976b6ee56e016112a5034ad4 WatchSource:0}: Error finding container 4425a59bfdfbaa52f2a12b25829ab9be2b8aae13976b6ee56e016112a5034ad4: Status 404 returned error can't find the container with id 4425a59bfdfbaa52f2a12b25829ab9be2b8aae13976b6ee56e016112a5034ad4 Jul 06 18:19:45.257814 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:45.257770 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27d5cb4d_462b_42ab_a879_fc2b7ae008b5.slice/crio-199e63615d8279397c6c48d7f3a6fe345ffcc54fccbf900de07992bf763771da WatchSource:0}: Error finding container 199e63615d8279397c6c48d7f3a6fe345ffcc54fccbf900de07992bf763771da: Status 404 returned error can't find the container with id 199e63615d8279397c6c48d7f3a6fe345ffcc54fccbf900de07992bf763771da Jul 06 18:19:45.283616 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:45.283589 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3157c562-b33e-4c9c-b724-cdac9220bbb0-original-pull-secret\") pod \"global-pull-secret-syncer-5hbx6\" (UID: \"3157c562-b33e-4c9c-b724-cdac9220bbb0\") " pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:19:45.283740 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:45.283726 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 18:19:45.283791 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:45.283782 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3157c562-b33e-4c9c-b724-cdac9220bbb0-original-pull-secret podName:3157c562-b33e-4c9c-b724-cdac9220bbb0 nodeName:}" failed. No retries permitted until 2026-07-06 18:19:46.283766334 +0000 UTC m=+3.010195276 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3157c562-b33e-4c9c-b724-cdac9220bbb0-original-pull-secret") pod "global-pull-secret-syncer-5hbx6" (UID: "3157c562-b33e-4c9c-b724-cdac9220bbb0") : object "kube-system"/"original-pull-secret" not registered Jul 06 18:19:45.384887 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:45.384855 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2c962e6-4904-4e1b-89f8-b6acb40f6a80-metrics-certs\") pod \"network-metrics-daemon-4gmr8\" (UID: \"f2c962e6-4904-4e1b-89f8-b6acb40f6a80\") " pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:19:45.385107 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:45.384906 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvx8j\" (UniqueName: \"kubernetes.io/projected/f142192b-8eb7-419d-bfc0-a97171b652b7-kube-api-access-fvx8j\") pod \"network-check-target-n4j4l\" (UID: \"f142192b-8eb7-419d-bfc0-a97171b652b7\") " pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:19:45.385107 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:45.385021 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:19:45.385107 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:45.385028 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 18:19:45.385107 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:45.385047 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 18:19:45.385107 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:45.385056 2566 projected.go:196] Error preparing data for projected volume kube-api-access-fvx8j for pod openshift-network-diagnostics/network-check-target-n4j4l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:19:45.385107 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:45.385087 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2c962e6-4904-4e1b-89f8-b6acb40f6a80-metrics-certs podName:f2c962e6-4904-4e1b-89f8-b6acb40f6a80 nodeName:}" failed. No retries permitted until 2026-07-06 18:19:46.385070606 +0000 UTC m=+3.111499756 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f2c962e6-4904-4e1b-89f8-b6acb40f6a80-metrics-certs") pod "network-metrics-daemon-4gmr8" (UID: "f2c962e6-4904-4e1b-89f8-b6acb40f6a80") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:19:45.385107 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:45.385103 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f142192b-8eb7-419d-bfc0-a97171b652b7-kube-api-access-fvx8j podName:f142192b-8eb7-419d-bfc0-a97171b652b7 nodeName:}" failed. No retries permitted until 2026-07-06 18:19:46.385096081 +0000 UTC m=+3.111525019 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fvx8j" (UniqueName: "kubernetes.io/projected/f142192b-8eb7-419d-bfc0-a97171b652b7-kube-api-access-fvx8j") pod "network-check-target-n4j4l" (UID: "f142192b-8eb7-419d-bfc0-a97171b652b7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:19:45.444112 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:19:45.444084 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc61cfa3e_6fda_4abf_8245_baeb901eb1f8.slice/crio-cac44ad7dbbb986a2396a69d6fe6235ed1922a61b388aa6baef15a91debbfd68 WatchSource:0}: Error finding container cac44ad7dbbb986a2396a69d6fe6235ed1922a61b388aa6baef15a91debbfd68: Status 404 returned error can't find the container with id cac44ad7dbbb986a2396a69d6fe6235ed1922a61b388aa6baef15a91debbfd68 Jul 06 18:19:45.714681 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:45.714579 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 18:14:45 +0000 UTC" deadline="2028-01-20 05:22:13.340832737 +0000 UTC" Jul 06 18:19:45.714681 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:45.714625 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13499h2m27.6262127s" Jul 06 18:19:45.843900 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:45.843851 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:19:45.844102 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:45.844024 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5hbx6" podUID="3157c562-b33e-4c9c-b724-cdac9220bbb0" Jul 06 18:19:45.844480 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:45.844461 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:19:45.844590 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:45.844569 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4j4l" podUID="f142192b-8eb7-419d-bfc0-a97171b652b7" Jul 06 18:19:45.844819 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:45.844799 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:19:45.844933 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:45.844911 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gmr8" podUID="f2c962e6-4904-4e1b-89f8-b6acb40f6a80" Jul 06 18:19:45.852833 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:45.852772 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqn7x" event={"ID":"afc02035-3216-4c87-971b-d0cc3cb1006d","Type":"ContainerStarted","Data":"9690159a1997e05e5b8c30b7acc5dfb0df6d000ad0a5d731fa1bce84ad47f1ff"} Jul 06 18:19:45.860412 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:45.860371 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-98.ec2.internal" event={"ID":"b7ccbef026f2cc3be335a3fabd6f8311","Type":"ContainerStarted","Data":"074b8e53ef0f3f58a9c4e8dda0eb01c1713ec97bfa4d24960e3cafec39f2f09d"} Jul 06 18:19:45.866502 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:45.866464 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" event={"ID":"c61cfa3e-6fda-4abf-8245-baeb901eb1f8","Type":"ContainerStarted","Data":"cac44ad7dbbb986a2396a69d6fe6235ed1922a61b388aa6baef15a91debbfd68"} Jul 06 18:19:45.871216 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:45.871132 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xczs7" event={"ID":"27d5cb4d-462b-42ab-a879-fc2b7ae008b5","Type":"ContainerStarted","Data":"199e63615d8279397c6c48d7f3a6fe345ffcc54fccbf900de07992bf763771da"} Jul 06 18:19:45.878406 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:45.878372 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" event={"ID":"1ede441d-79f2-4365-8a08-403bf043ef73","Type":"ContainerStarted","Data":"d9148c8cf12caff9dfafb26bcef475a43e63ea1379afa700910c403534ce4b09"} Jul 06 18:19:45.892986 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:45.892899 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ldrkc" event={"ID":"ee4e9134-c963-43ab-923a-42caaa210f43","Type":"ContainerStarted","Data":"06531132804b51e8b02c699914361b402c2e96b9e470317104d9e31da7e6bdc6"} Jul 06 18:19:45.904720 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:45.904677 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-98.ec2.internal" event={"ID":"ff73c77bfdbc9702b903107bb5e1e802","Type":"ContainerStarted","Data":"3662f980179a9e3e5f952b8636afce3971c34349e723d245a7b7132b72ed9c44"} Jul 06 18:19:45.911965 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:45.911885 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cz6db" event={"ID":"29a00562-9fe6-473d-a4fd-31df8896cac8","Type":"ContainerStarted","Data":"4425a59bfdfbaa52f2a12b25829ab9be2b8aae13976b6ee56e016112a5034ad4"} Jul 06 18:19:45.913312 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:45.913286 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vttwc" event={"ID":"2a97d8c3-ea18-47d0-a315-59c693911c46","Type":"ContainerStarted","Data":"a31be7049697abf51f69d964db3c570da4e4ba38babcfba3578aa5fe42fc561b"} Jul 06 18:19:45.914400 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:45.914378 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r8tl5" event={"ID":"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f","Type":"ContainerStarted","Data":"cc1672a988ccb67ccca3553ec17e1dc470eff5b23ed295d5182d2432640ac1fd"} Jul 06 18:19:45.923407 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:45.923349 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c2wvn" event={"ID":"991513c4-afc5-4f9a-b436-648417e2b0e4","Type":"ContainerStarted","Data":"c22067df9e964ed2a8bff8cf33c4383c13ac48167523bac64689a48b4eb2da36"} Jul 06 18:19:46.058128 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:46.058043 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 18:19:46.225029 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:46.224988 2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 18:19:46.293248 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:46.293206 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3157c562-b33e-4c9c-b724-cdac9220bbb0-original-pull-secret\") pod \"global-pull-secret-syncer-5hbx6\" (UID: \"3157c562-b33e-4c9c-b724-cdac9220bbb0\") " pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:19:46.293415 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:46.293399 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 18:19:46.293476 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:46.293464 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3157c562-b33e-4c9c-b724-cdac9220bbb0-original-pull-secret podName:3157c562-b33e-4c9c-b724-cdac9220bbb0 nodeName:}" failed. No retries permitted until 2026-07-06 18:19:48.293445759 +0000 UTC m=+5.019874715 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3157c562-b33e-4c9c-b724-cdac9220bbb0-original-pull-secret") pod "global-pull-secret-syncer-5hbx6" (UID: "3157c562-b33e-4c9c-b724-cdac9220bbb0") : object "kube-system"/"original-pull-secret" not registered Jul 06 18:19:46.394740 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:46.394646 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2c962e6-4904-4e1b-89f8-b6acb40f6a80-metrics-certs\") pod \"network-metrics-daemon-4gmr8\" (UID: \"f2c962e6-4904-4e1b-89f8-b6acb40f6a80\") " pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:19:46.394740 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:46.394711 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvx8j\" (UniqueName: \"kubernetes.io/projected/f142192b-8eb7-419d-bfc0-a97171b652b7-kube-api-access-fvx8j\") pod \"network-check-target-n4j4l\" (UID: \"f142192b-8eb7-419d-bfc0-a97171b652b7\") " pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:19:46.395016 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:46.394921 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 18:19:46.395016 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:46.394942 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 18:19:46.395016 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:46.394957 2566 projected.go:196] Error preparing data for projected volume kube-api-access-fvx8j for pod openshift-network-diagnostics/network-check-target-n4j4l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:19:46.395182 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:46.395021 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f142192b-8eb7-419d-bfc0-a97171b652b7-kube-api-access-fvx8j podName:f142192b-8eb7-419d-bfc0-a97171b652b7 nodeName:}" failed. No retries permitted until 2026-07-06 18:19:48.395001757 +0000 UTC m=+5.121430720 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fvx8j" (UniqueName: "kubernetes.io/projected/f142192b-8eb7-419d-bfc0-a97171b652b7-kube-api-access-fvx8j") pod "network-check-target-n4j4l" (UID: "f142192b-8eb7-419d-bfc0-a97171b652b7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:19:46.395182 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:46.395028 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:19:46.395182 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:46.395098 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2c962e6-4904-4e1b-89f8-b6acb40f6a80-metrics-certs podName:f2c962e6-4904-4e1b-89f8-b6acb40f6a80 nodeName:}" failed. No retries permitted until 2026-07-06 18:19:48.395079264 +0000 UTC m=+5.121508206 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f2c962e6-4904-4e1b-89f8-b6acb40f6a80-metrics-certs") pod "network-metrics-daemon-4gmr8" (UID: "f2c962e6-4904-4e1b-89f8-b6acb40f6a80") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:19:46.714859 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:46.714754 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 18:14:45 +0000 UTC" deadline="2028-01-19 09:12:28.272614252 +0000 UTC" Jul 06 18:19:46.714859 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:46.714801 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13478h52m41.557818974s" Jul 06 18:19:47.849228 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:47.849145 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:19:47.849707 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:47.849294 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gmr8" podUID="f2c962e6-4904-4e1b-89f8-b6acb40f6a80" Jul 06 18:19:47.849774 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:47.849761 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:19:47.850419 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:47.849861 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5hbx6" podUID="3157c562-b33e-4c9c-b724-cdac9220bbb0" Jul 06 18:19:47.850419 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:47.849931 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:19:47.850419 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:47.850254 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4j4l" podUID="f142192b-8eb7-419d-bfc0-a97171b652b7" Jul 06 18:19:48.313420 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:48.313324 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3157c562-b33e-4c9c-b724-cdac9220bbb0-original-pull-secret\") pod \"global-pull-secret-syncer-5hbx6\" (UID: \"3157c562-b33e-4c9c-b724-cdac9220bbb0\") " pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:19:48.313588 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:48.313477 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 18:19:48.313588 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:48.313538 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3157c562-b33e-4c9c-b724-cdac9220bbb0-original-pull-secret podName:3157c562-b33e-4c9c-b724-cdac9220bbb0 nodeName:}" failed. No retries permitted until 2026-07-06 18:19:52.313515759 +0000 UTC m=+9.039944704 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3157c562-b33e-4c9c-b724-cdac9220bbb0-original-pull-secret") pod "global-pull-secret-syncer-5hbx6" (UID: "3157c562-b33e-4c9c-b724-cdac9220bbb0") : object "kube-system"/"original-pull-secret" not registered Jul 06 18:19:48.414595 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:48.414547 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2c962e6-4904-4e1b-89f8-b6acb40f6a80-metrics-certs\") pod \"network-metrics-daemon-4gmr8\" (UID: \"f2c962e6-4904-4e1b-89f8-b6acb40f6a80\") " pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:19:48.414793 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:48.414612 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvx8j\" (UniqueName: \"kubernetes.io/projected/f142192b-8eb7-419d-bfc0-a97171b652b7-kube-api-access-fvx8j\") pod \"network-check-target-n4j4l\" (UID: \"f142192b-8eb7-419d-bfc0-a97171b652b7\") " pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:19:48.415130 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:48.414807 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 18:19:48.415130 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:48.414826 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 18:19:48.415130 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:48.414840 2566 projected.go:196] Error preparing data for projected volume kube-api-access-fvx8j for pod openshift-network-diagnostics/network-check-target-n4j4l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:19:48.415130 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:48.414913 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f142192b-8eb7-419d-bfc0-a97171b652b7-kube-api-access-fvx8j podName:f142192b-8eb7-419d-bfc0-a97171b652b7 nodeName:}" failed. No retries permitted until 2026-07-06 18:19:52.414893307 +0000 UTC m=+9.141322264 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fvx8j" (UniqueName: "kubernetes.io/projected/f142192b-8eb7-419d-bfc0-a97171b652b7-kube-api-access-fvx8j") pod "network-check-target-n4j4l" (UID: "f142192b-8eb7-419d-bfc0-a97171b652b7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:19:48.415370 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:48.415321 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:19:48.415370 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:48.415363 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2c962e6-4904-4e1b-89f8-b6acb40f6a80-metrics-certs podName:f2c962e6-4904-4e1b-89f8-b6acb40f6a80 nodeName:}" failed. No retries permitted until 2026-07-06 18:19:52.41535031 +0000 UTC m=+9.141779248 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f2c962e6-4904-4e1b-89f8-b6acb40f6a80-metrics-certs") pod "network-metrics-daemon-4gmr8" (UID: "f2c962e6-4904-4e1b-89f8-b6acb40f6a80") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:19:49.844663 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:49.844006 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:19:49.844663 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:49.844149 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4j4l" podUID="f142192b-8eb7-419d-bfc0-a97171b652b7" Jul 06 18:19:49.844663 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:49.844495 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:19:49.844663 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:49.844587 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5hbx6" podUID="3157c562-b33e-4c9c-b724-cdac9220bbb0" Jul 06 18:19:49.845570 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:49.845407 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:19:49.845570 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:49.845531 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gmr8" podUID="f2c962e6-4904-4e1b-89f8-b6acb40f6a80" Jul 06 18:19:51.846646 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:51.846566 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:19:51.846646 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:51.846591 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:19:51.847172 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:51.846566 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:19:51.847172 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:51.846701 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gmr8" podUID="f2c962e6-4904-4e1b-89f8-b6acb40f6a80" Jul 06 18:19:51.847172 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:51.846805 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5hbx6" podUID="3157c562-b33e-4c9c-b724-cdac9220bbb0" Jul 06 18:19:51.847172 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:51.846920 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4j4l" podUID="f142192b-8eb7-419d-bfc0-a97171b652b7" Jul 06 18:19:52.351197 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:52.351152 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3157c562-b33e-4c9c-b724-cdac9220bbb0-original-pull-secret\") pod \"global-pull-secret-syncer-5hbx6\" (UID: \"3157c562-b33e-4c9c-b724-cdac9220bbb0\") " pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:19:52.351396 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:52.351317 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 18:19:52.351396 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:52.351387 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3157c562-b33e-4c9c-b724-cdac9220bbb0-original-pull-secret podName:3157c562-b33e-4c9c-b724-cdac9220bbb0 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:00.351371704 +0000 UTC m=+17.077800642 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3157c562-b33e-4c9c-b724-cdac9220bbb0-original-pull-secret") pod "global-pull-secret-syncer-5hbx6" (UID: "3157c562-b33e-4c9c-b724-cdac9220bbb0") : object "kube-system"/"original-pull-secret" not registered Jul 06 18:19:52.452599 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:52.452559 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2c962e6-4904-4e1b-89f8-b6acb40f6a80-metrics-certs\") pod \"network-metrics-daemon-4gmr8\" (UID: \"f2c962e6-4904-4e1b-89f8-b6acb40f6a80\") " pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:19:52.452599 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:52.452612 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvx8j\" (UniqueName: \"kubernetes.io/projected/f142192b-8eb7-419d-bfc0-a97171b652b7-kube-api-access-fvx8j\") pod \"network-check-target-n4j4l\" (UID: \"f142192b-8eb7-419d-bfc0-a97171b652b7\") " pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:19:52.452881 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:52.452731 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:19:52.452881 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:52.452746 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 18:19:52.452881 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:52.452764 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 18:19:52.452881 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:52.452776 2566 projected.go:196] Error preparing data for projected volume kube-api-access-fvx8j for pod openshift-network-diagnostics/network-check-target-n4j4l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:19:52.452881 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:52.452801 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2c962e6-4904-4e1b-89f8-b6acb40f6a80-metrics-certs podName:f2c962e6-4904-4e1b-89f8-b6acb40f6a80 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:00.452779979 +0000 UTC m=+17.179208939 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f2c962e6-4904-4e1b-89f8-b6acb40f6a80-metrics-certs") pod "network-metrics-daemon-4gmr8" (UID: "f2c962e6-4904-4e1b-89f8-b6acb40f6a80") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:19:52.452881 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:52.452819 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f142192b-8eb7-419d-bfc0-a97171b652b7-kube-api-access-fvx8j podName:f142192b-8eb7-419d-bfc0-a97171b652b7 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:00.452807786 +0000 UTC m=+17.179236724 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fvx8j" (UniqueName: "kubernetes.io/projected/f142192b-8eb7-419d-bfc0-a97171b652b7-kube-api-access-fvx8j") pod "network-check-target-n4j4l" (UID: "f142192b-8eb7-419d-bfc0-a97171b652b7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:19:53.844122 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:53.844080 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:19:53.844631 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:53.844216 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5hbx6" podUID="3157c562-b33e-4c9c-b724-cdac9220bbb0" Jul 06 18:19:53.844631 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:53.844623 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:19:53.844731 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:53.844703 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4j4l" podUID="f142192b-8eb7-419d-bfc0-a97171b652b7" Jul 06 18:19:53.844790 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:53.844766 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:19:53.844875 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:53.844856 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gmr8" podUID="f2c962e6-4904-4e1b-89f8-b6acb40f6a80" Jul 06 18:19:55.843149 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:55.843112 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:19:55.843593 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:55.843241 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5hbx6" podUID="3157c562-b33e-4c9c-b724-cdac9220bbb0" Jul 06 18:19:55.843593 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:55.843549 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:19:55.843722 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:55.843614 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4j4l" podUID="f142192b-8eb7-419d-bfc0-a97171b652b7" Jul 06 18:19:55.843780 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:55.843753 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:19:55.843943 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:55.843901 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gmr8" podUID="f2c962e6-4904-4e1b-89f8-b6acb40f6a80" Jul 06 18:19:57.844342 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:57.843827 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:19:57.844342 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:57.843857 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:19:57.844342 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:57.843832 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:19:57.844342 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:57.843974 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4j4l" podUID="f142192b-8eb7-419d-bfc0-a97171b652b7" Jul 06 18:19:57.844342 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:57.844087 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gmr8" podUID="f2c962e6-4904-4e1b-89f8-b6acb40f6a80" Jul 06 18:19:57.844342 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:57.844164 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5hbx6" podUID="3157c562-b33e-4c9c-b724-cdac9220bbb0" Jul 06 18:19:59.843394 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:59.843354 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:19:59.843860 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:59.843395 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:19:59.843860 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:19:59.843476 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:19:59.843860 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:59.843487 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4j4l" podUID="f142192b-8eb7-419d-bfc0-a97171b652b7" Jul 06 18:19:59.843860 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:59.843566 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5hbx6" podUID="3157c562-b33e-4c9c-b724-cdac9220bbb0" Jul 06 18:19:59.843860 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:19:59.843658 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gmr8" podUID="f2c962e6-4904-4e1b-89f8-b6acb40f6a80" Jul 06 18:20:00.417932 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:00.417893 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3157c562-b33e-4c9c-b724-cdac9220bbb0-original-pull-secret\") pod \"global-pull-secret-syncer-5hbx6\" (UID: \"3157c562-b33e-4c9c-b724-cdac9220bbb0\") " pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:20:00.418185 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:00.418054 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 18:20:00.418185 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:00.418124 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3157c562-b33e-4c9c-b724-cdac9220bbb0-original-pull-secret podName:3157c562-b33e-4c9c-b724-cdac9220bbb0 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:16.41810498 +0000 UTC m=+33.144533916 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3157c562-b33e-4c9c-b724-cdac9220bbb0-original-pull-secret") pod "global-pull-secret-syncer-5hbx6" (UID: "3157c562-b33e-4c9c-b724-cdac9220bbb0") : object "kube-system"/"original-pull-secret" not registered Jul 06 18:20:00.518779 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:00.518744 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvx8j\" (UniqueName: \"kubernetes.io/projected/f142192b-8eb7-419d-bfc0-a97171b652b7-kube-api-access-fvx8j\") pod \"network-check-target-n4j4l\" (UID: \"f142192b-8eb7-419d-bfc0-a97171b652b7\") " pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:20:00.518969 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:00.518821 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2c962e6-4904-4e1b-89f8-b6acb40f6a80-metrics-certs\") pod \"network-metrics-daemon-4gmr8\" (UID: \"f2c962e6-4904-4e1b-89f8-b6acb40f6a80\") " pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:20:00.518969 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:00.518918 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:20:00.518969 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:00.518947 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 18:20:00.519119 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:00.518971 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 18:20:00.519119 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:00.518984 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2c962e6-4904-4e1b-89f8-b6acb40f6a80-metrics-certs podName:f2c962e6-4904-4e1b-89f8-b6acb40f6a80 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:16.518965213 +0000 UTC m=+33.245394162 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f2c962e6-4904-4e1b-89f8-b6acb40f6a80-metrics-certs") pod "network-metrics-daemon-4gmr8" (UID: "f2c962e6-4904-4e1b-89f8-b6acb40f6a80") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:20:00.519119 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:00.518987 2566 projected.go:196] Error preparing data for projected volume kube-api-access-fvx8j for pod openshift-network-diagnostics/network-check-target-n4j4l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:20:00.519119 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:00.519039 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f142192b-8eb7-419d-bfc0-a97171b652b7-kube-api-access-fvx8j podName:f142192b-8eb7-419d-bfc0-a97171b652b7 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:16.519022308 +0000 UTC m=+33.245451252 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fvx8j" (UniqueName: "kubernetes.io/projected/f142192b-8eb7-419d-bfc0-a97171b652b7-kube-api-access-fvx8j") pod "network-check-target-n4j4l" (UID: "f142192b-8eb7-419d-bfc0-a97171b652b7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:20:01.843513 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:01.843435 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:20:01.843915 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:01.843433 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:20:01.843915 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:01.843566 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4j4l" podUID="f142192b-8eb7-419d-bfc0-a97171b652b7" Jul 06 18:20:01.843915 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:01.843589 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:20:01.843915 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:01.843692 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5hbx6" podUID="3157c562-b33e-4c9c-b724-cdac9220bbb0" Jul 06 18:20:01.843915 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:01.843787 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gmr8" podUID="f2c962e6-4904-4e1b-89f8-b6acb40f6a80" Jul 06 18:20:03.844259 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:03.844229 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:20:03.844669 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:03.844316 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5hbx6" podUID="3157c562-b33e-4c9c-b724-cdac9220bbb0" Jul 06 18:20:03.844669 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:03.844406 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:20:03.844669 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:03.844514 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4j4l" podUID="f142192b-8eb7-419d-bfc0-a97171b652b7" Jul 06 18:20:03.844669 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:03.844563 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:20:03.844669 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:03.844647 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gmr8" podUID="f2c962e6-4904-4e1b-89f8-b6acb40f6a80" Jul 06 18:20:05.843560 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:05.843519 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:20:05.844067 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:05.843522 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:20:05.844067 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:05.843637 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5hbx6" podUID="3157c562-b33e-4c9c-b724-cdac9220bbb0" Jul 06 18:20:05.844067 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:05.843531 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:20:05.844067 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:05.843701 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4j4l" podUID="f142192b-8eb7-419d-bfc0-a97171b652b7" Jul 06 18:20:05.844067 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:05.843805 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gmr8" podUID="f2c962e6-4904-4e1b-89f8-b6acb40f6a80" Jul 06 18:20:06.977428 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:06.977035 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cz6db" event={"ID":"29a00562-9fe6-473d-a4fd-31df8896cac8","Type":"ContainerStarted","Data":"86a0304f8a0452b0098053379cea5dbb418aa062c82e59028bfdd096b985516b"} Jul 06 18:20:06.978528 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:06.978499 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-98.ec2.internal" event={"ID":"b7ccbef026f2cc3be335a3fabd6f8311","Type":"ContainerStarted","Data":"1f8bc2a4224632519e72a1bc33ece906c1e0f834852102ab413141753766d9c1"} Jul 06 18:20:06.979905 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:06.979859 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" event={"ID":"c61cfa3e-6fda-4abf-8245-baeb901eb1f8","Type":"ContainerStarted","Data":"5cc3c50b9dec9907f195f87568888fe7c4b70cfdc93b0775d73aef4b39e1fa46"} Jul 06 18:20:06.982770 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:06.982750 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffzjr_1ede441d-79f2-4365-8a08-403bf043ef73/ovn-acl-logging/0.log" Jul 06 18:20:06.983139 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:06.983119 2566 generic.go:356] "Generic (PLEG): container finished" podID="1ede441d-79f2-4365-8a08-403bf043ef73" containerID="f4b1f3bb605ff1840a54f9e370c6890d828919298567b4ad02b444704e920b4e" exitCode=1 Jul 06 18:20:06.983200 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:06.983152 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" event={"ID":"1ede441d-79f2-4365-8a08-403bf043ef73","Type":"ContainerStarted","Data":"4171b86397be48a0f024c5ff5f80c40346fd673d27c2a0e2d408e2c2a3276774"} Jul 06 18:20:06.983200 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:06.983197 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" event={"ID":"1ede441d-79f2-4365-8a08-403bf043ef73","Type":"ContainerStarted","Data":"ff888fff40849595982ce9c69e7cce9a2c2e3ff37d1930da28b5b2c1492de4be"} Jul 06 18:20:06.983276 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:06.983227 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" event={"ID":"1ede441d-79f2-4365-8a08-403bf043ef73","Type":"ContainerStarted","Data":"309f57ba1d00e28f430c8c726811cba5df81bb7e26684673c063ab439393fbb3"} Jul 06 18:20:06.983276 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:06.983253 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" event={"ID":"1ede441d-79f2-4365-8a08-403bf043ef73","Type":"ContainerStarted","Data":"258b10173411cb3250627aae44166920db87da96484a809ee5cedb3562133d5e"} Jul 06 18:20:06.983342 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:06.983281 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" event={"ID":"1ede441d-79f2-4365-8a08-403bf043ef73","Type":"ContainerDied","Data":"f4b1f3bb605ff1840a54f9e370c6890d828919298567b4ad02b444704e920b4e"} Jul 06 18:20:06.983342 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:06.983309 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" event={"ID":"1ede441d-79f2-4365-8a08-403bf043ef73","Type":"ContainerStarted","Data":"89fa9db67e3400436a8f091c04b3414b088655efae95e3f1f3335dae48af0d55"} Jul 06 18:20:06.994132 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:06.994091 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-cz6db" podStartSLOduration=1.930878283 podStartE2EDuration="22.994078913s" totalImagesPullingTime="21.06320063s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:19:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:06.993727184 +0000 UTC m=+23.720156146" watchObservedRunningTime="2026-07-06 18:20:06.994078913 +0000 UTC m=+23.720507870" Jul 06 18:20:07.023094 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:07.023047 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-98.ec2.internal" podStartSLOduration=22.023030338 podStartE2EDuration="22.023030338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:19:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:07.007473437 +0000 UTC m=+23.733902396" watchObservedRunningTime="2026-07-06 18:20:07.023030338 +0000 UTC m=+23.749459295" Jul 06 18:20:07.023259 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:07.023163 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-kvr5q" podStartSLOduration=2.3614842879999998 podStartE2EDuration="23.023157216s" totalImagesPullingTime="20.661672928s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:19:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:07.02274609 +0000 UTC m=+23.749175049" watchObservedRunningTime="2026-07-06 18:20:07.023157216 +0000 UTC m=+23.749586174" Jul 06 18:20:07.843260 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:07.843211 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:20:07.843260 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:07.843226 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:20:07.843260 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:07.843244 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:20:07.843640 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:07.843340 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4j4l" podUID="f142192b-8eb7-419d-bfc0-a97171b652b7" Jul 06 18:20:07.843640 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:07.843452 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gmr8" podUID="f2c962e6-4904-4e1b-89f8-b6acb40f6a80" Jul 06 18:20:07.843640 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:07.843521 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5hbx6" podUID="3157c562-b33e-4c9c-b724-cdac9220bbb0" Jul 06 18:20:07.987265 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:07.987228 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c2wvn" event={"ID":"991513c4-afc5-4f9a-b436-648417e2b0e4","Type":"ContainerStarted","Data":"5ede9184c0c76d0909de588ba51c339cd92ed641634c5a43e7e778614e270b79"} Jul 06 18:20:07.989023 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:07.988980 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqn7x" event={"ID":"afc02035-3216-4c87-971b-d0cc3cb1006d","Type":"ContainerStarted","Data":"2d9e9813d36ef382f32b7121591fe058bb1482742f70ea833a6100960a163353"} Jul 06 18:20:07.990601 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:07.990573 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xczs7" event={"ID":"27d5cb4d-462b-42ab-a879-fc2b7ae008b5","Type":"ContainerStarted","Data":"15aa60490de131b9db1b7987e76b879d6d43ef82a19c763c4eca57b615e5f3b4"} Jul 06 18:20:07.992188 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:07.992159 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ldrkc" event={"ID":"ee4e9134-c963-43ab-923a-42caaa210f43","Type":"ContainerStarted","Data":"91e1b5ce7a8f5944b21b0d369e3844fa09db4f053529a4f5393a811897841e6b"} Jul 06 18:20:07.994097 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:07.994071 2566 generic.go:356] "Generic (PLEG): container finished" podID="ff73c77bfdbc9702b903107bb5e1e802" containerID="83e59e6942cf2a2c0ed9533595aca0c47a7809c2673a078b2e808af7956e89af" exitCode=0 Jul 06 18:20:07.994199 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:07.994151 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-98.ec2.internal" event={"ID":"ff73c77bfdbc9702b903107bb5e1e802","Type":"ContainerDied","Data":"83e59e6942cf2a2c0ed9533595aca0c47a7809c2673a078b2e808af7956e89af"} Jul 06 18:20:07.996948 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:07.996922 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vttwc" event={"ID":"2a97d8c3-ea18-47d0-a315-59c693911c46","Type":"ContainerStarted","Data":"10955c75ea78d03ff8d137430d3328e09ec7bb9034d62cc6dea00d2b687b9090"} Jul 06 18:20:07.998751 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:07.998715 2566 generic.go:356] "Generic (PLEG): container finished" podID="adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f" containerID="0bfc42cab790b44fa4de066bb1ac8cfcf4aecd0df92245cd7634e18490122c24" exitCode=0 Jul 06 18:20:07.998837 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:07.998813 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r8tl5" event={"ID":"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f","Type":"ContainerDied","Data":"0bfc42cab790b44fa4de066bb1ac8cfcf4aecd0df92245cd7634e18490122c24"} Jul 06 18:20:08.020034 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:08.019983 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-c2wvn" podStartSLOduration=3.164087993 podStartE2EDuration="24.019967364s" totalImagesPullingTime="20.855879371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:19:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:08.00496533 +0000 UTC m=+24.731394291" watchObservedRunningTime="2026-07-06 18:20:08.019967364 +0000 UTC m=+24.746396321" Jul 06 18:20:08.033841 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:08.033799 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-ldrkc" podStartSLOduration=2.934255884 podStartE2EDuration="24.033783436s" totalImagesPullingTime="21.099527552s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:19:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:08.033470693 +0000 UTC m=+24.759899651" watchObservedRunningTime="2026-07-06 18:20:08.033783436 +0000 UTC m=+24.760212394" Jul 06 18:20:08.068961 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:08.068917 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xczs7" podStartSLOduration=3.449724842 podStartE2EDuration="24.06889979s" totalImagesPullingTime="20.619174948s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:19:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:08.068583239 +0000 UTC m=+24.795012255" watchObservedRunningTime="2026-07-06 18:20:08.06889979 +0000 UTC m=+24.795328745" Jul 06 18:20:08.087757 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:08.087711 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vttwc" podStartSLOduration=3.021243577 podStartE2EDuration="24.087695787s" totalImagesPullingTime="21.06645221s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:19:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:08.087226068 +0000 UTC m=+24.813655027" watchObservedRunningTime="2026-07-06 18:20:08.087695787 +0000 UTC m=+24.814124745" Jul 06 18:20:08.179154 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:08.179115 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:08.251063 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:08.251017 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:08.757959 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:08.757763 2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 18:20:09.003092 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:09.003058 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqn7x" event={"ID":"afc02035-3216-4c87-971b-d0cc3cb1006d","Type":"ContainerStarted","Data":"40d750684092d2a49bed0086b3661adaf82add8d4fc1479d587c1557b05d55b5"} Jul 06 18:20:09.006048 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:09.006028 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffzjr_1ede441d-79f2-4365-8a08-403bf043ef73/ovn-acl-logging/0.log" Jul 06 18:20:09.006413 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:09.006394 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" event={"ID":"1ede441d-79f2-4365-8a08-403bf043ef73","Type":"ContainerStarted","Data":"6bdc11863ca5c2f89c15048891a7876af2e000e7781955b7f189e654be0e1f08"} Jul 06 18:20:09.008197 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:09.008165 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-98.ec2.internal" event={"ID":"ff73c77bfdbc9702b903107bb5e1e802","Type":"ContainerStarted","Data":"254179ae7bd50662f19c86a72e914a3841350a48cde7e841525ed17591c55dae"} Jul 06 18:20:09.025801 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:09.025743 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-98.ec2.internal" podStartSLOduration=24.025723574 podStartE2EDuration="24.025723574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:19:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:09.02555703 +0000 UTC m=+25.751985989" watchObservedRunningTime="2026-07-06 18:20:09.025723574 +0000 UTC m=+25.752152533" Jul 06 18:20:09.745001 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:09.744909 2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T18:20:08.75792393Z","UUID":"ad612dd0-2d7c-4ed0-9788-a31e5def18a9","Handler":null,"Name":"","Endpoint":""} Jul 06 18:20:09.746690 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:09.746663 2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 18:20:09.746816 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:09.746706 2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 18:20:09.843600 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:09.843566 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:20:09.843775 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:09.843567 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:20:09.843775 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:09.843700 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4j4l" podUID="f142192b-8eb7-419d-bfc0-a97171b652b7" Jul 06 18:20:09.843921 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:09.843785 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5hbx6" podUID="3157c562-b33e-4c9c-b724-cdac9220bbb0" Jul 06 18:20:09.843921 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:09.843839 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:20:09.844008 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:09.843932 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gmr8" podUID="f2c962e6-4904-4e1b-89f8-b6acb40f6a80" Jul 06 18:20:10.012997 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:10.012901 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqn7x" event={"ID":"afc02035-3216-4c87-971b-d0cc3cb1006d","Type":"ContainerStarted","Data":"2bf0386a6c053810441abd887c0296ff4f5a3c3152515b1c3dec312ea885f174"} Jul 06 18:20:10.033356 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:10.033308 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hqn7x" podStartSLOduration=2.77103258 podStartE2EDuration="26.033292934s" totalImagesPullingTime="23.262260354s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:19:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:10.032879124 +0000 UTC m=+26.759308073" watchObservedRunningTime="2026-07-06 18:20:10.033292934 +0000 UTC m=+26.759721921" Jul 06 18:20:10.141424 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:10.141393 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-c2wvn" Jul 06 18:20:10.141975 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:10.141956 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-c2wvn" Jul 06 18:20:10.146509 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:10.146487 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:10.214911 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:10.214852 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:11.015811 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:11.015619 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-c2wvn" Jul 06 18:20:11.016196 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:11.016185 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-c2wvn" Jul 06 18:20:11.089176 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:11.088950 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:11.843654 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:11.843561 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:20:11.843810 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:11.843561 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:20:11.843810 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:11.843694 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4j4l" podUID="f142192b-8eb7-419d-bfc0-a97171b652b7" Jul 06 18:20:11.843810 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:11.843752 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5hbx6" podUID="3157c562-b33e-4c9c-b724-cdac9220bbb0" Jul 06 18:20:11.843810 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:11.843561 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:20:11.844005 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:11.843850 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gmr8" podUID="f2c962e6-4904-4e1b-89f8-b6acb40f6a80" Jul 06 18:20:12.020967 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:12.020938 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffzjr_1ede441d-79f2-4365-8a08-403bf043ef73/ovn-acl-logging/0.log" Jul 06 18:20:12.021388 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:12.021337 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" event={"ID":"1ede441d-79f2-4365-8a08-403bf043ef73","Type":"ContainerStarted","Data":"46c747c58fb4454e22bb0305a13212f64aee488e8e98beddc7a409ba15bb8aef"} Jul 06 18:20:12.021769 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:12.021754 2566 scope.go:117] "RemoveContainer" containerID="f4b1f3bb605ff1840a54f9e370c6890d828919298567b4ad02b444704e920b4e" Jul 06 18:20:12.806186 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:12.806153 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:12.907579 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:12.907545 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:13.027900 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:13.027855 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffzjr_1ede441d-79f2-4365-8a08-403bf043ef73/ovn-acl-logging/0.log" Jul 06 18:20:13.028314 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:13.028294 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" event={"ID":"1ede441d-79f2-4365-8a08-403bf043ef73","Type":"ContainerStarted","Data":"6b98774e30e7022488c3b1d9c9858b709bd772ebb7e8cf208d29541e8828c3d6"} Jul 06 18:20:13.028510 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:13.028491 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:20:13.028647 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:13.028550 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:20:13.028647 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:13.028596 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:20:13.047906 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:13.047860 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:20:13.048066 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:13.047991 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:20:13.059719 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:13.059675 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" podStartSLOduration=7.87850281 podStartE2EDuration="29.059660835s" totalImagesPullingTime="21.181158025s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:19:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:13.058036565 +0000 UTC m=+29.784465527" watchObservedRunningTime="2026-07-06 18:20:13.059660835 +0000 UTC m=+29.786089830" Jul 06 18:20:13.273093 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:13.272833 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5hbx6"] Jul 06 18:20:13.273229 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:13.273205 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:20:13.273321 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:13.273300 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5hbx6" podUID="3157c562-b33e-4c9c-b724-cdac9220bbb0" Jul 06 18:20:13.276516 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:13.276478 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n4j4l"] Jul 06 18:20:13.276649 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:13.276541 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4gmr8"] Jul 06 18:20:13.276649 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:13.276627 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:20:13.276743 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:13.276718 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gmr8" podUID="f2c962e6-4904-4e1b-89f8-b6acb40f6a80" Jul 06 18:20:13.276811 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:13.276796 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:20:13.276933 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:13.276913 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4j4l" podUID="f142192b-8eb7-419d-bfc0-a97171b652b7" Jul 06 18:20:13.772601 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:13.772559 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:13.871798 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:13.871765 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:14.451704 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:14.451666 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:14.509681 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:14.509649 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:14.843303 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:14.843227 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:20:14.843303 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:14.843256 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:20:14.843303 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:14.843256 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:20:14.843521 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:14.843371 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gmr8" podUID="f2c962e6-4904-4e1b-89f8-b6acb40f6a80" Jul 06 18:20:14.843521 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:14.843494 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5hbx6" podUID="3157c562-b33e-4c9c-b724-cdac9220bbb0" Jul 06 18:20:14.843607 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:14.843591 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4j4l" podUID="f142192b-8eb7-419d-bfc0-a97171b652b7" Jul 06 18:20:15.034189 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:15.034157 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:15.140897 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:15.140842 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:15.200688 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:15.200651 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:15.257972 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:15.257938 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:15.468093 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:15.468012 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:15.570413 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:15.570385 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:15.756513 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:15.756432 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:15.852952 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:15.852858 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:15.952171 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:15.952128 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:16.036919 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:16.036889 2566 generic.go:356] "Generic (PLEG): container finished" podID="adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f" containerID="40c7d9fef2b47b3e721fa2d67209dd5ddf88f5713b4135d233e432a2c9397479" exitCode=0 Jul 06 18:20:16.037059 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:16.036940 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r8tl5" event={"ID":"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f","Type":"ContainerDied","Data":"40c7d9fef2b47b3e721fa2d67209dd5ddf88f5713b4135d233e432a2c9397479"} Jul 06 18:20:16.228098 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:16.228070 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vttwc_2a97d8c3-ea18-47d0-a315-59c693911c46/dns-node-resolver/0.log" Jul 06 18:20:16.435600 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:16.435565 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3157c562-b33e-4c9c-b724-cdac9220bbb0-original-pull-secret\") pod \"global-pull-secret-syncer-5hbx6\" (UID: \"3157c562-b33e-4c9c-b724-cdac9220bbb0\") " pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:20:16.435896 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:16.435684 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 18:20:16.435896 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:16.435763 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3157c562-b33e-4c9c-b724-cdac9220bbb0-original-pull-secret podName:3157c562-b33e-4c9c-b724-cdac9220bbb0 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:48.435743793 +0000 UTC m=+65.162172742 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3157c562-b33e-4c9c-b724-cdac9220bbb0-original-pull-secret") pod "global-pull-secret-syncer-5hbx6" (UID: "3157c562-b33e-4c9c-b724-cdac9220bbb0") : object "kube-system"/"original-pull-secret" not registered Jul 06 18:20:16.536467 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:16.536429 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2c962e6-4904-4e1b-89f8-b6acb40f6a80-metrics-certs\") pod \"network-metrics-daemon-4gmr8\" (UID: \"f2c962e6-4904-4e1b-89f8-b6acb40f6a80\") " pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:20:16.537215 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:16.536476 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvx8j\" (UniqueName: \"kubernetes.io/projected/f142192b-8eb7-419d-bfc0-a97171b652b7-kube-api-access-fvx8j\") pod \"network-check-target-n4j4l\" (UID: \"f142192b-8eb7-419d-bfc0-a97171b652b7\") " pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:20:16.537215 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:16.536541 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:20:16.537215 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:16.536621 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2c962e6-4904-4e1b-89f8-b6acb40f6a80-metrics-certs podName:f2c962e6-4904-4e1b-89f8-b6acb40f6a80 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:48.536600962 +0000 UTC m=+65.263029901 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f2c962e6-4904-4e1b-89f8-b6acb40f6a80-metrics-certs") pod "network-metrics-daemon-4gmr8" (UID: "f2c962e6-4904-4e1b-89f8-b6acb40f6a80") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 18:20:16.537215 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:16.536631 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 18:20:16.537215 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:16.536652 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 18:20:16.537215 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:16.536662 2566 projected.go:196] Error preparing data for projected volume kube-api-access-fvx8j for pod openshift-network-diagnostics/network-check-target-n4j4l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:20:16.537215 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:16.536713 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f142192b-8eb7-419d-bfc0-a97171b652b7-kube-api-access-fvx8j podName:f142192b-8eb7-419d-bfc0-a97171b652b7 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:48.536701326 +0000 UTC m=+65.263130262 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fvx8j" (UniqueName: "kubernetes.io/projected/f142192b-8eb7-419d-bfc0-a97171b652b7-kube-api-access-fvx8j") pod "network-check-target-n4j4l" (UID: "f142192b-8eb7-419d-bfc0-a97171b652b7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 18:20:16.842998 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:16.842919 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:20:16.843140 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:16.842920 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:20:16.843140 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:16.843040 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gmr8" podUID="f2c962e6-4904-4e1b-89f8-b6acb40f6a80" Jul 06 18:20:16.843140 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:16.843051 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:20:16.843140 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:16.843126 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5hbx6" podUID="3157c562-b33e-4c9c-b724-cdac9220bbb0" Jul 06 18:20:16.843341 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:16.843182 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4j4l" podUID="f142192b-8eb7-419d-bfc0-a97171b652b7" Jul 06 18:20:17.227885 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:17.227851 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xczs7_27d5cb4d-462b-42ab-a879-fc2b7ae008b5/node-ca/0.log" Jul 06 18:20:17.378188 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:17.378162 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:17.432262 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:17.432234 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:18.043761 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:18.043544 2566 generic.go:356] "Generic (PLEG): container finished" podID="adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f" containerID="5f5e3a55a6e05d5ca4c9fc3dabc42589417096c138897dac0568aac4dd0d50ba" exitCode=0 Jul 06 18:20:18.044153 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:18.043632 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r8tl5" event={"ID":"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f","Type":"ContainerDied","Data":"5f5e3a55a6e05d5ca4c9fc3dabc42589417096c138897dac0568aac4dd0d50ba"} Jul 06 18:20:18.115912 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:18.115878 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:18.249657 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:18.249628 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:18.843820 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:18.843783 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:20:18.843820 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:18.843813 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:20:18.843988 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:18.843929 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n4j4l" podUID="f142192b-8eb7-419d-bfc0-a97171b652b7" Jul 06 18:20:18.843988 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:18.843979 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:20:18.844100 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:18.844077 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4gmr8" podUID="f2c962e6-4904-4e1b-89f8-b6acb40f6a80" Jul 06 18:20:18.844166 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:18.844149 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5hbx6" podUID="3157c562-b33e-4c9c-b724-cdac9220bbb0" Jul 06 18:20:19.076419 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:19.076342 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-98.ec2.internal" event="NodeReady" Jul 06 18:20:19.076885 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:19.076488 2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 18:20:20.050803 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:20.050774 2566 generic.go:356] "Generic (PLEG): container finished" podID="adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f" containerID="dbabf267192cc8694883ed39c6eb50e06792067963f61be6c236bc1842c203c3" exitCode=0 Jul 06 18:20:20.050961 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:20.050821 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r8tl5" event={"ID":"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f","Type":"ContainerDied","Data":"dbabf267192cc8694883ed39c6eb50e06792067963f61be6c236bc1842c203c3"} Jul 06 18:20:20.843909 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:20.843854 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:20:20.844452 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:20.843918 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:20:20.844452 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:20.843888 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:20:20.848928 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:20.848901 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 18:20:20.849074 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:20.848954 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 18:20:20.849144 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:20.849118 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 18:20:20.849144 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:20.849129 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rgvgt\"" Jul 06 18:20:20.849240 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:20.849143 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 18:20:20.849240 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:20.849226 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mpmrs\"" Jul 06 18:20:21.149347 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:21.149315 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:21.380107 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:21.380077 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:21.514799 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:21.514688 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:21.615887 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:21.615849 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:21.834577 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:21.834497 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:21.935230 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:21.935193 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:22.092724 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:22.092631 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:23.292527 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:23.292471 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:23.484191 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:23.484165 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:23.681457 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:23.681422 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:23.877044 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:23.877006 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:24.083321 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:24.083240 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:24.207380 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:24.207347 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:24.309017 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:24.308985 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:24.480493 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:24.480462 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:25.017502 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:25.017464 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:25.120491 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:25.120452 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:25.219325 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:25.219269 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:25.483489 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:25.483458 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:25.690013 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:25.689984 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:25.881650 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:25.881622 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:25.985257 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:25.985225 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:26.068588 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:26.068495 2566 generic.go:356] "Generic (PLEG): container finished" podID="adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f" containerID="8f0430f952b5aaab12a49f2e384d8bed39608afba3b78e54668505efa9856c8c" exitCode=0 Jul 06 18:20:26.068588 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:26.068533 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r8tl5" event={"ID":"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f","Type":"ContainerDied","Data":"8f0430f952b5aaab12a49f2e384d8bed39608afba3b78e54668505efa9856c8c"} Jul 06 18:20:26.211096 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:26.211060 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:27.073726 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:27.073689 2566 generic.go:356] "Generic (PLEG): container finished" podID="adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f" containerID="64add8641ee6fc54bcbdaefd0c3867617a8c625a5716c85208c71fedb241041d" exitCode=0 Jul 06 18:20:27.074130 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:27.073748 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r8tl5" event={"ID":"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f","Type":"ContainerDied","Data":"64add8641ee6fc54bcbdaefd0c3867617a8c625a5716c85208c71fedb241041d"} Jul 06 18:20:27.078808 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:27.078784 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:27.179903 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:27.179857 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:27.279658 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:27.279626 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:27.477118 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:27.477074 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:27.586426 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:27.586395 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:27.685531 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:27.685502 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:27.882746 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:27.882660 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:28.081965 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:28.081931 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r8tl5" event={"ID":"adc427cd-3561-4ff2-b61a-fe4fbe4cdf7f","Type":"ContainerStarted","Data":"b3b06ce9cdc28def80dfd90956cb977d07381702383f90ca40c3567375ba1710"} Jul 06 18:20:28.109745 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:28.109688 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-r8tl5" podStartSLOduration=7.425729841 podStartE2EDuration="44.109671464s" totalImagesPullingTime="36.683941623s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:19:44 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:28.109290464 +0000 UTC m=+44.835719626" watchObservedRunningTime="2026-07-06 18:20:28.109671464 +0000 UTC m=+44.836100429" Jul 06 18:20:28.283137 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:28.283105 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:28.882099 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:28.882070 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:29.282463 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:29.282432 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:29.701656 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:29.701629 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:29.893475 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:29.893438 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:30.279193 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:30.279158 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:30.883012 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:30.882978 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:31.293250 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:31.293218 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:31.686100 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:31.686050 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:31.879509 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:31.879471 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:32.079274 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:32.079195 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:35.283425 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:35.283390 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:35.382427 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:35.382399 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:35.561443 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:35.561378 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:35.945457 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:35.945420 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:36.049173 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.049137 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:36.137935 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.137898 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:36.172580 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.172546 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-clcjg"] Jul 06 18:20:36.174799 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.174783 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-clcjg" Jul 06 18:20:36.178006 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.177975 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 18:20:36.178006 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.177988 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 18:20:36.178185 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.178012 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-f7c2d\"" Jul 06 18:20:36.178185 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.177988 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 18:20:36.178185 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.177987 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 18:20:36.180293 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.180268 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4chpk"] Jul 06 18:20:36.182318 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.182299 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4chpk" Jul 06 18:20:36.184155 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.184132 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6cfff669f4-6fmtq"] Jul 06 18:20:36.185892 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.185857 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dkzn2\"" Jul 06 18:20:36.186104 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.186087 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 18:20:36.186229 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.186142 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 18:20:36.186229 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.186192 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 18:20:36.186415 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.186400 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" Jul 06 18:20:36.188793 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.188773 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-clcjg"] Jul 06 18:20:36.189415 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.189385 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 18:20:36.189781 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.189763 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 18:20:36.189853 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.189784 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 18:20:36.189853 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.189808 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-h42sk\"" Jul 06 18:20:36.196698 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.196640 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 18:20:36.196788 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.196743 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4chpk"] Jul 06 18:20:36.201521 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.201497 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cfff669f4-6fmtq"] Jul 06 18:20:36.231296 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.231262 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:36.277565 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.277530 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e26e29f4-5957-4076-ab8b-3f741af97a3c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-clcjg\" (UID: \"e26e29f4-5957-4076-ab8b-3f741af97a3c\") " pod="openshift-insights/insights-runtime-extractor-clcjg" Jul 06 18:20:36.277565 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.277570 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/056b10b5-54ff-452a-94b2-24134a58f725-image-registry-private-configuration\") pod \"image-registry-6cfff669f4-6fmtq\" (UID: \"056b10b5-54ff-452a-94b2-24134a58f725\") " pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" Jul 06 18:20:36.277832 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.277605 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz88m\" (UniqueName: \"kubernetes.io/projected/e26e29f4-5957-4076-ab8b-3f741af97a3c-kube-api-access-kz88m\") pod \"insights-runtime-extractor-clcjg\" (UID: \"e26e29f4-5957-4076-ab8b-3f741af97a3c\") " pod="openshift-insights/insights-runtime-extractor-clcjg" Jul 06 18:20:36.277832 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.277786 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/056b10b5-54ff-452a-94b2-24134a58f725-registry-tls\") pod \"image-registry-6cfff669f4-6fmtq\" (UID: \"056b10b5-54ff-452a-94b2-24134a58f725\") " pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" Jul 06 18:20:36.277976 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.277882 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wz7d\" (UniqueName: \"kubernetes.io/projected/056b10b5-54ff-452a-94b2-24134a58f725-kube-api-access-8wz7d\") pod \"image-registry-6cfff669f4-6fmtq\" (UID: \"056b10b5-54ff-452a-94b2-24134a58f725\") " pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" Jul 06 18:20:36.277976 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.277947 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn2jn\" (UniqueName: \"kubernetes.io/projected/01ea6ede-0121-412d-b17c-5d9073fbb347-kube-api-access-vn2jn\") pod \"ingress-canary-4chpk\" (UID: \"01ea6ede-0121-412d-b17c-5d9073fbb347\") " pod="openshift-ingress-canary/ingress-canary-4chpk" Jul 06 18:20:36.278075 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.277989 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e26e29f4-5957-4076-ab8b-3f741af97a3c-crio-socket\") pod \"insights-runtime-extractor-clcjg\" (UID: \"e26e29f4-5957-4076-ab8b-3f741af97a3c\") " pod="openshift-insights/insights-runtime-extractor-clcjg" Jul 06 18:20:36.278075 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.278014 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/056b10b5-54ff-452a-94b2-24134a58f725-ca-trust-extracted\") pod \"image-registry-6cfff669f4-6fmtq\" (UID: \"056b10b5-54ff-452a-94b2-24134a58f725\") " pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" Jul 06 18:20:36.278171 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.278100 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e26e29f4-5957-4076-ab8b-3f741af97a3c-data-volume\") pod \"insights-runtime-extractor-clcjg\" (UID: \"e26e29f4-5957-4076-ab8b-3f741af97a3c\") " pod="openshift-insights/insights-runtime-extractor-clcjg" Jul 06 18:20:36.278171 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.278125 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/056b10b5-54ff-452a-94b2-24134a58f725-registry-certificates\") pod \"image-registry-6cfff669f4-6fmtq\" (UID: \"056b10b5-54ff-452a-94b2-24134a58f725\") " pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" Jul 06 18:20:36.278171 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.278155 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/056b10b5-54ff-452a-94b2-24134a58f725-trusted-ca\") pod \"image-registry-6cfff669f4-6fmtq\" (UID: \"056b10b5-54ff-452a-94b2-24134a58f725\") " pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" Jul 06 18:20:36.278276 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.278207 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e26e29f4-5957-4076-ab8b-3f741af97a3c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-clcjg\" (UID: \"e26e29f4-5957-4076-ab8b-3f741af97a3c\") " pod="openshift-insights/insights-runtime-extractor-clcjg" Jul 06 18:20:36.278276 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.278251 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/056b10b5-54ff-452a-94b2-24134a58f725-installation-pull-secrets\") pod \"image-registry-6cfff669f4-6fmtq\" (UID: \"056b10b5-54ff-452a-94b2-24134a58f725\") " pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" Jul 06 18:20:36.278354 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.278279 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/056b10b5-54ff-452a-94b2-24134a58f725-bound-sa-token\") pod \"image-registry-6cfff669f4-6fmtq\" (UID: \"056b10b5-54ff-452a-94b2-24134a58f725\") " pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" Jul 06 18:20:36.278354 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.278302 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01ea6ede-0121-412d-b17c-5d9073fbb347-cert\") pod \"ingress-canary-4chpk\" (UID: \"01ea6ede-0121-412d-b17c-5d9073fbb347\") " pod="openshift-ingress-canary/ingress-canary-4chpk" Jul 06 18:20:36.278522 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.278506 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rn4lz"] Jul 06 18:20:36.283954 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.283924 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rn4lz" Jul 06 18:20:36.286990 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.286965 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 18:20:36.287244 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.287219 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 18:20:36.287450 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.287435 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9wdv2\"" Jul 06 18:20:36.291667 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.291645 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rn4lz"] Jul 06 18:20:36.328766 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.328733 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:36.378997 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.378958 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e26e29f4-5957-4076-ab8b-3f741af97a3c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-clcjg\" (UID: \"e26e29f4-5957-4076-ab8b-3f741af97a3c\") " pod="openshift-insights/insights-runtime-extractor-clcjg" Jul 06 18:20:36.378997 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.378996 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/056b10b5-54ff-452a-94b2-24134a58f725-image-registry-private-configuration\") pod \"image-registry-6cfff669f4-6fmtq\" (UID: \"056b10b5-54ff-452a-94b2-24134a58f725\") " pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" Jul 06 18:20:36.379247 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.379030 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz88m\" (UniqueName: \"kubernetes.io/projected/e26e29f4-5957-4076-ab8b-3f741af97a3c-kube-api-access-kz88m\") pod \"insights-runtime-extractor-clcjg\" (UID: \"e26e29f4-5957-4076-ab8b-3f741af97a3c\") " pod="openshift-insights/insights-runtime-extractor-clcjg" Jul 06 18:20:36.379247 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.379055 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/056b10b5-54ff-452a-94b2-24134a58f725-registry-tls\") pod \"image-registry-6cfff669f4-6fmtq\" (UID: \"056b10b5-54ff-452a-94b2-24134a58f725\") " pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" Jul 06 18:20:36.379350 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.379235 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wz7d\" (UniqueName: \"kubernetes.io/projected/056b10b5-54ff-452a-94b2-24134a58f725-kube-api-access-8wz7d\") pod \"image-registry-6cfff669f4-6fmtq\" (UID: \"056b10b5-54ff-452a-94b2-24134a58f725\") " pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" Jul 06 18:20:36.379350 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.379288 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8987aa45-6cac-4458-9d82-3a4e9c85e09e-tmp-dir\") pod \"dns-default-rn4lz\" (UID: \"8987aa45-6cac-4458-9d82-3a4e9c85e09e\") " pod="openshift-dns/dns-default-rn4lz" Jul 06 18:20:36.379350 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.379316 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8987aa45-6cac-4458-9d82-3a4e9c85e09e-metrics-tls\") pod \"dns-default-rn4lz\" (UID: \"8987aa45-6cac-4458-9d82-3a4e9c85e09e\") " pod="openshift-dns/dns-default-rn4lz" Jul 06 18:20:36.379477 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.379430 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpxbt\" (UniqueName: \"kubernetes.io/projected/8987aa45-6cac-4458-9d82-3a4e9c85e09e-kube-api-access-zpxbt\") pod \"dns-default-rn4lz\" (UID: \"8987aa45-6cac-4458-9d82-3a4e9c85e09e\") " pod="openshift-dns/dns-default-rn4lz" Jul 06 18:20:36.379477 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.379464 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn2jn\" (UniqueName: \"kubernetes.io/projected/01ea6ede-0121-412d-b17c-5d9073fbb347-kube-api-access-vn2jn\") pod \"ingress-canary-4chpk\" (UID: \"01ea6ede-0121-412d-b17c-5d9073fbb347\") " pod="openshift-ingress-canary/ingress-canary-4chpk" Jul 06 18:20:36.379559 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.379493 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e26e29f4-5957-4076-ab8b-3f741af97a3c-crio-socket\") pod \"insights-runtime-extractor-clcjg\" (UID: \"e26e29f4-5957-4076-ab8b-3f741af97a3c\") " pod="openshift-insights/insights-runtime-extractor-clcjg" Jul 06 18:20:36.379559 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.379515 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/056b10b5-54ff-452a-94b2-24134a58f725-ca-trust-extracted\") pod \"image-registry-6cfff669f4-6fmtq\" (UID: \"056b10b5-54ff-452a-94b2-24134a58f725\") " pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" Jul 06 18:20:36.379559 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.379544 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8987aa45-6cac-4458-9d82-3a4e9c85e09e-config-volume\") pod \"dns-default-rn4lz\" (UID: \"8987aa45-6cac-4458-9d82-3a4e9c85e09e\") " pod="openshift-dns/dns-default-rn4lz" Jul 06 18:20:36.379694 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.379610 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e26e29f4-5957-4076-ab8b-3f741af97a3c-data-volume\") pod \"insights-runtime-extractor-clcjg\" (UID: \"e26e29f4-5957-4076-ab8b-3f741af97a3c\") " pod="openshift-insights/insights-runtime-extractor-clcjg" Jul 06 18:20:36.379694 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.379631 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/056b10b5-54ff-452a-94b2-24134a58f725-registry-certificates\") pod \"image-registry-6cfff669f4-6fmtq\" (UID: \"056b10b5-54ff-452a-94b2-24134a58f725\") " pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" Jul 06 18:20:36.379694 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.379662 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/056b10b5-54ff-452a-94b2-24134a58f725-trusted-ca\") pod \"image-registry-6cfff669f4-6fmtq\" (UID: \"056b10b5-54ff-452a-94b2-24134a58f725\") " pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" Jul 06 18:20:36.379694 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.379667 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e26e29f4-5957-4076-ab8b-3f741af97a3c-crio-socket\") pod \"insights-runtime-extractor-clcjg\" (UID: \"e26e29f4-5957-4076-ab8b-3f741af97a3c\") " pod="openshift-insights/insights-runtime-extractor-clcjg" Jul 06 18:20:36.379694 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.379692 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e26e29f4-5957-4076-ab8b-3f741af97a3c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-clcjg\" (UID: \"e26e29f4-5957-4076-ab8b-3f741af97a3c\") " pod="openshift-insights/insights-runtime-extractor-clcjg" Jul 06 18:20:36.379949 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.379739 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/056b10b5-54ff-452a-94b2-24134a58f725-installation-pull-secrets\") pod \"image-registry-6cfff669f4-6fmtq\" (UID: \"056b10b5-54ff-452a-94b2-24134a58f725\") " pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" Jul 06 18:20:36.379949 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.379765 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/056b10b5-54ff-452a-94b2-24134a58f725-bound-sa-token\") pod \"image-registry-6cfff669f4-6fmtq\" (UID: \"056b10b5-54ff-452a-94b2-24134a58f725\") " pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" Jul 06 18:20:36.379949 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.379788 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01ea6ede-0121-412d-b17c-5d9073fbb347-cert\") pod \"ingress-canary-4chpk\" (UID: \"01ea6ede-0121-412d-b17c-5d9073fbb347\") " pod="openshift-ingress-canary/ingress-canary-4chpk" Jul 06 18:20:36.380061 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.379945 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/056b10b5-54ff-452a-94b2-24134a58f725-ca-trust-extracted\") pod \"image-registry-6cfff669f4-6fmtq\" (UID: \"056b10b5-54ff-452a-94b2-24134a58f725\") " pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" Jul 06 18:20:36.380061 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.379992 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e26e29f4-5957-4076-ab8b-3f741af97a3c-data-volume\") pod \"insights-runtime-extractor-clcjg\" (UID: \"e26e29f4-5957-4076-ab8b-3f741af97a3c\") " pod="openshift-insights/insights-runtime-extractor-clcjg" Jul 06 18:20:36.380598 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.380571 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/056b10b5-54ff-452a-94b2-24134a58f725-registry-certificates\") pod \"image-registry-6cfff669f4-6fmtq\" (UID: \"056b10b5-54ff-452a-94b2-24134a58f725\") " pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" Jul 06 18:20:36.380710 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.380694 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/056b10b5-54ff-452a-94b2-24134a58f725-trusted-ca\") pod \"image-registry-6cfff669f4-6fmtq\" (UID: \"056b10b5-54ff-452a-94b2-24134a58f725\") " pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" Jul 06 18:20:36.380844 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.380821 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e26e29f4-5957-4076-ab8b-3f741af97a3c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-clcjg\" (UID: \"e26e29f4-5957-4076-ab8b-3f741af97a3c\") " pod="openshift-insights/insights-runtime-extractor-clcjg" Jul 06 18:20:36.383725 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.383700 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/056b10b5-54ff-452a-94b2-24134a58f725-installation-pull-secrets\") pod \"image-registry-6cfff669f4-6fmtq\" (UID: \"056b10b5-54ff-452a-94b2-24134a58f725\") " pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" Jul 06 18:20:36.383798 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.383733 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01ea6ede-0121-412d-b17c-5d9073fbb347-cert\") pod \"ingress-canary-4chpk\" (UID: \"01ea6ede-0121-412d-b17c-5d9073fbb347\") " pod="openshift-ingress-canary/ingress-canary-4chpk" Jul 06 18:20:36.383798 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.383756 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e26e29f4-5957-4076-ab8b-3f741af97a3c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-clcjg\" (UID: \"e26e29f4-5957-4076-ab8b-3f741af97a3c\") " pod="openshift-insights/insights-runtime-extractor-clcjg" Jul 06 18:20:36.383798 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.383790 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/056b10b5-54ff-452a-94b2-24134a58f725-image-registry-private-configuration\") pod \"image-registry-6cfff669f4-6fmtq\" (UID: \"056b10b5-54ff-452a-94b2-24134a58f725\") " pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" Jul 06 18:20:36.383926 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.383858 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/056b10b5-54ff-452a-94b2-24134a58f725-registry-tls\") pod \"image-registry-6cfff669f4-6fmtq\" (UID: \"056b10b5-54ff-452a-94b2-24134a58f725\") " pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" Jul 06 18:20:36.390452 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.390406 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wz7d\" (UniqueName: \"kubernetes.io/projected/056b10b5-54ff-452a-94b2-24134a58f725-kube-api-access-8wz7d\") pod \"image-registry-6cfff669f4-6fmtq\" (UID: \"056b10b5-54ff-452a-94b2-24134a58f725\") " pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" Jul 06 18:20:36.390826 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.390803 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn2jn\" (UniqueName: \"kubernetes.io/projected/01ea6ede-0121-412d-b17c-5d9073fbb347-kube-api-access-vn2jn\") pod \"ingress-canary-4chpk\" (UID: \"01ea6ede-0121-412d-b17c-5d9073fbb347\") " pod="openshift-ingress-canary/ingress-canary-4chpk" Jul 06 18:20:36.390986 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.390964 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/056b10b5-54ff-452a-94b2-24134a58f725-bound-sa-token\") pod \"image-registry-6cfff669f4-6fmtq\" (UID: \"056b10b5-54ff-452a-94b2-24134a58f725\") " pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" Jul 06 18:20:36.391053 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.391014 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz88m\" (UniqueName: \"kubernetes.io/projected/e26e29f4-5957-4076-ab8b-3f741af97a3c-kube-api-access-kz88m\") pod \"insights-runtime-extractor-clcjg\" (UID: \"e26e29f4-5957-4076-ab8b-3f741af97a3c\") " pod="openshift-insights/insights-runtime-extractor-clcjg" Jul 06 18:20:36.428859 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.428824 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:36.480497 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.480405 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8987aa45-6cac-4458-9d82-3a4e9c85e09e-tmp-dir\") pod \"dns-default-rn4lz\" (UID: \"8987aa45-6cac-4458-9d82-3a4e9c85e09e\") " pod="openshift-dns/dns-default-rn4lz" Jul 06 18:20:36.480497 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.480444 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8987aa45-6cac-4458-9d82-3a4e9c85e09e-metrics-tls\") pod \"dns-default-rn4lz\" (UID: \"8987aa45-6cac-4458-9d82-3a4e9c85e09e\") " pod="openshift-dns/dns-default-rn4lz" Jul 06 18:20:36.480497 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.480473 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpxbt\" (UniqueName: \"kubernetes.io/projected/8987aa45-6cac-4458-9d82-3a4e9c85e09e-kube-api-access-zpxbt\") pod \"dns-default-rn4lz\" (UID: \"8987aa45-6cac-4458-9d82-3a4e9c85e09e\") " pod="openshift-dns/dns-default-rn4lz" Jul 06 18:20:36.480497 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.480500 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8987aa45-6cac-4458-9d82-3a4e9c85e09e-config-volume\") pod \"dns-default-rn4lz\" (UID: \"8987aa45-6cac-4458-9d82-3a4e9c85e09e\") " pod="openshift-dns/dns-default-rn4lz" Jul 06 18:20:36.480845 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.480822 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8987aa45-6cac-4458-9d82-3a4e9c85e09e-tmp-dir\") pod \"dns-default-rn4lz\" (UID: \"8987aa45-6cac-4458-9d82-3a4e9c85e09e\") " pod="openshift-dns/dns-default-rn4lz" Jul 06 18:20:36.481098 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.481081 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8987aa45-6cac-4458-9d82-3a4e9c85e09e-config-volume\") pod \"dns-default-rn4lz\" (UID: \"8987aa45-6cac-4458-9d82-3a4e9c85e09e\") " pod="openshift-dns/dns-default-rn4lz" Jul 06 18:20:36.482960 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.482945 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8987aa45-6cac-4458-9d82-3a4e9c85e09e-metrics-tls\") pod \"dns-default-rn4lz\" (UID: \"8987aa45-6cac-4458-9d82-3a4e9c85e09e\") " pod="openshift-dns/dns-default-rn4lz" Jul 06 18:20:36.483850 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.483835 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-clcjg" Jul 06 18:20:36.489957 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.489933 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpxbt\" (UniqueName: \"kubernetes.io/projected/8987aa45-6cac-4458-9d82-3a4e9c85e09e-kube-api-access-zpxbt\") pod \"dns-default-rn4lz\" (UID: \"8987aa45-6cac-4458-9d82-3a4e9c85e09e\") " pod="openshift-dns/dns-default-rn4lz" Jul 06 18:20:36.494886 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.494850 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4chpk" Jul 06 18:20:36.500102 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.500079 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" Jul 06 18:20:36.527846 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.527808 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:36.593177 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.593150 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rn4lz" Jul 06 18:20:36.642449 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.642418 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4chpk"] Jul 06 18:20:36.656617 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.656319 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-clcjg"] Jul 06 18:20:36.659101 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:20:36.659023 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode26e29f4_5957_4076_ab8b_3f741af97a3c.slice/crio-4f1a5088b9a326d3a75c4b68a97437550fb37ffee23220f5e33dca86c106d098 WatchSource:0}: Error finding container 4f1a5088b9a326d3a75c4b68a97437550fb37ffee23220f5e33dca86c106d098: Status 404 returned error can't find the container with id 4f1a5088b9a326d3a75c4b68a97437550fb37ffee23220f5e33dca86c106d098 Jul 06 18:20:36.659101 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.659041 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:36.681825 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.681767 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cfff669f4-6fmtq"] Jul 06 18:20:36.684729 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:20:36.684633 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod056b10b5_54ff_452a_94b2_24134a58f725.slice/crio-ca2ec10e3c473d838bbcb52d6f53fc314e34db6f1003bdf90c7052c6781ad10d WatchSource:0}: Error finding container ca2ec10e3c473d838bbcb52d6f53fc314e34db6f1003bdf90c7052c6781ad10d: Status 404 returned error can't find the container with id ca2ec10e3c473d838bbcb52d6f53fc314e34db6f1003bdf90c7052c6781ad10d Jul 06 18:20:36.719666 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.719629 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:36.738019 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.737939 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rn4lz"] Jul 06 18:20:36.741557 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:20:36.741530 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8987aa45_6cac_4458_9d82_3a4e9c85e09e.slice/crio-6ba5a896df5d525ec8e39d4ca1e4e96257c0129526a8524add42c55806509910 WatchSource:0}: Error finding container 6ba5a896df5d525ec8e39d4ca1e4e96257c0129526a8524add42c55806509910: Status 404 returned error can't find the container with id 6ba5a896df5d525ec8e39d4ca1e4e96257c0129526a8524add42c55806509910 Jul 06 18:20:36.804601 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.804566 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:36.956972 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:36.956854 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:37.025115 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:37.025039 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:37.085305 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:37.085267 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:37.105457 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:37.105405 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rn4lz" event={"ID":"8987aa45-6cac-4458-9d82-3a4e9c85e09e","Type":"ContainerStarted","Data":"6ba5a896df5d525ec8e39d4ca1e4e96257c0129526a8524add42c55806509910"} Jul 06 18:20:37.107667 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:37.107635 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4chpk" event={"ID":"01ea6ede-0121-412d-b17c-5d9073fbb347","Type":"ContainerStarted","Data":"d97e1ceac545126f9a09edb2812f94bf4d612e4fdb462539190d54d2a2aa176c"} Jul 06 18:20:37.109661 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:37.109341 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-clcjg" event={"ID":"e26e29f4-5957-4076-ab8b-3f741af97a3c","Type":"ContainerStarted","Data":"4f1a5088b9a326d3a75c4b68a97437550fb37ffee23220f5e33dca86c106d098"} Jul 06 18:20:37.111339 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:37.111297 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" event={"ID":"056b10b5-54ff-452a-94b2-24134a58f725","Type":"ContainerStarted","Data":"9e4517ff5a02edd05524fd30c81cd7cc81ff85e100485b6cac70c80faf51c61c"} Jul 06 18:20:37.111455 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:37.111344 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" event={"ID":"056b10b5-54ff-452a-94b2-24134a58f725","Type":"ContainerStarted","Data":"ca2ec10e3c473d838bbcb52d6f53fc314e34db6f1003bdf90c7052c6781ad10d"} Jul 06 18:20:37.111455 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:37.111440 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" Jul 06 18:20:37.133359 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:37.133297 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" podStartSLOduration=1.133279608 podStartE2EDuration="1.133279608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:20:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:37.13296953 +0000 UTC m=+53.859398489" watchObservedRunningTime="2026-07-06 18:20:37.133279608 +0000 UTC m=+53.859708544" Jul 06 18:20:37.371816 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:37.371727 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:37.472538 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:37.472503 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:37.534691 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:37.534615 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:38.259916 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:38.259881 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:38.651604 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:38.651572 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:38.700393 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:38.700339 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:38.778289 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:38.778242 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:38.839240 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:38.839207 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:39.468001 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:39.467964 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:39.571164 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:39.571135 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:39.617186 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:39.617147 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jqslw"] Jul 06 18:20:39.629432 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:39.629407 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jqslw" Jul 06 18:20:39.632353 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:39.632293 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jqslw"] Jul 06 18:20:39.632777 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:39.632757 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-kqh5r\"" Jul 06 18:20:39.632777 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:39.632772 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 06 18:20:39.648079 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:39.648047 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:39.716021 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:39.715988 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0d0bd084-df2e-4a5e-93ec-4142ba42228a-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-jqslw\" (UID: \"0d0bd084-df2e-4a5e-93ec-4142ba42228a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jqslw" Jul 06 18:20:39.716436 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:39.716075 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:39.817003 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:39.816898 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0d0bd084-df2e-4a5e-93ec-4142ba42228a-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-jqslw\" (UID: \"0d0bd084-df2e-4a5e-93ec-4142ba42228a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jqslw" Jul 06 18:20:39.819813 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:39.819781 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0d0bd084-df2e-4a5e-93ec-4142ba42228a-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-jqslw\" (UID: \"0d0bd084-df2e-4a5e-93ec-4142ba42228a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jqslw" Jul 06 18:20:39.942776 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:39.942740 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jqslw" Jul 06 18:20:39.975699 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:39.975660 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:40.073068 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.072990 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:40.138885 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.138832 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:40.191645 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.191610 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6699bc67c9-zwmdh"] Jul 06 18:20:40.194332 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.194311 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6699bc67c9-zwmdh" Jul 06 18:20:40.198793 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.198766 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 18:20:40.198947 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.198811 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 18:20:40.198947 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.198849 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 18:20:40.199050 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.198767 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 18:20:40.199097 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.198766 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 18:20:40.199180 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.199160 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 18:20:40.199317 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.199289 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-xhrkm\"" Jul 06 18:20:40.199385 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.199344 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 18:20:40.201287 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.201264 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6699bc67c9-zwmdh"] Jul 06 18:20:40.205669 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.205645 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:40.219693 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.219665 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a62d81e-42fa-4604-ab20-11130d5cd553-service-ca\") pod \"console-6699bc67c9-zwmdh\" (UID: \"4a62d81e-42fa-4604-ab20-11130d5cd553\") " pod="openshift-console/console-6699bc67c9-zwmdh" Jul 06 18:20:40.219837 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.219706 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6pgs\" (UniqueName: \"kubernetes.io/projected/4a62d81e-42fa-4604-ab20-11130d5cd553-kube-api-access-f6pgs\") pod \"console-6699bc67c9-zwmdh\" (UID: \"4a62d81e-42fa-4604-ab20-11130d5cd553\") " pod="openshift-console/console-6699bc67c9-zwmdh" Jul 06 18:20:40.219837 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.219794 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a62d81e-42fa-4604-ab20-11130d5cd553-console-config\") pod \"console-6699bc67c9-zwmdh\" (UID: \"4a62d81e-42fa-4604-ab20-11130d5cd553\") " pod="openshift-console/console-6699bc67c9-zwmdh" Jul 06 18:20:40.219971 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.219836 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a62d81e-42fa-4604-ab20-11130d5cd553-console-oauth-config\") pod \"console-6699bc67c9-zwmdh\" (UID: \"4a62d81e-42fa-4604-ab20-11130d5cd553\") " pod="openshift-console/console-6699bc67c9-zwmdh" Jul 06 18:20:40.219971 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.219859 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a62d81e-42fa-4604-ab20-11130d5cd553-oauth-serving-cert\") pod \"console-6699bc67c9-zwmdh\" (UID: \"4a62d81e-42fa-4604-ab20-11130d5cd553\") " pod="openshift-console/console-6699bc67c9-zwmdh" Jul 06 18:20:40.220060 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.219977 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a62d81e-42fa-4604-ab20-11130d5cd553-console-serving-cert\") pod \"console-6699bc67c9-zwmdh\" (UID: \"4a62d81e-42fa-4604-ab20-11130d5cd553\") " pod="openshift-console/console-6699bc67c9-zwmdh" Jul 06 18:20:40.321203 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.321168 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a62d81e-42fa-4604-ab20-11130d5cd553-console-oauth-config\") pod \"console-6699bc67c9-zwmdh\" (UID: \"4a62d81e-42fa-4604-ab20-11130d5cd553\") " pod="openshift-console/console-6699bc67c9-zwmdh" Jul 06 18:20:40.321393 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.321211 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a62d81e-42fa-4604-ab20-11130d5cd553-oauth-serving-cert\") pod \"console-6699bc67c9-zwmdh\" (UID: \"4a62d81e-42fa-4604-ab20-11130d5cd553\") " pod="openshift-console/console-6699bc67c9-zwmdh" Jul 06 18:20:40.321393 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.321296 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a62d81e-42fa-4604-ab20-11130d5cd553-console-serving-cert\") pod \"console-6699bc67c9-zwmdh\" (UID: \"4a62d81e-42fa-4604-ab20-11130d5cd553\") " pod="openshift-console/console-6699bc67c9-zwmdh" Jul 06 18:20:40.321393 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.321323 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a62d81e-42fa-4604-ab20-11130d5cd553-service-ca\") pod \"console-6699bc67c9-zwmdh\" (UID: \"4a62d81e-42fa-4604-ab20-11130d5cd553\") " pod="openshift-console/console-6699bc67c9-zwmdh" Jul 06 18:20:40.321393 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.321346 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6pgs\" (UniqueName: \"kubernetes.io/projected/4a62d81e-42fa-4604-ab20-11130d5cd553-kube-api-access-f6pgs\") pod \"console-6699bc67c9-zwmdh\" (UID: \"4a62d81e-42fa-4604-ab20-11130d5cd553\") " pod="openshift-console/console-6699bc67c9-zwmdh" Jul 06 18:20:40.321598 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.321396 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a62d81e-42fa-4604-ab20-11130d5cd553-console-config\") pod \"console-6699bc67c9-zwmdh\" (UID: \"4a62d81e-42fa-4604-ab20-11130d5cd553\") " pod="openshift-console/console-6699bc67c9-zwmdh" Jul 06 18:20:40.322228 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.322197 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a62d81e-42fa-4604-ab20-11130d5cd553-oauth-serving-cert\") pod \"console-6699bc67c9-zwmdh\" (UID: \"4a62d81e-42fa-4604-ab20-11130d5cd553\") " pod="openshift-console/console-6699bc67c9-zwmdh" Jul 06 18:20:40.322471 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.322435 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a62d81e-42fa-4604-ab20-11130d5cd553-service-ca\") pod \"console-6699bc67c9-zwmdh\" (UID: \"4a62d81e-42fa-4604-ab20-11130d5cd553\") " pod="openshift-console/console-6699bc67c9-zwmdh" Jul 06 18:20:40.322945 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.322921 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a62d81e-42fa-4604-ab20-11130d5cd553-console-config\") pod \"console-6699bc67c9-zwmdh\" (UID: \"4a62d81e-42fa-4604-ab20-11130d5cd553\") " pod="openshift-console/console-6699bc67c9-zwmdh" Jul 06 18:20:40.324255 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.324203 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a62d81e-42fa-4604-ab20-11130d5cd553-console-serving-cert\") pod \"console-6699bc67c9-zwmdh\" (UID: \"4a62d81e-42fa-4604-ab20-11130d5cd553\") " pod="openshift-console/console-6699bc67c9-zwmdh" Jul 06 18:20:40.324255 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.324235 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a62d81e-42fa-4604-ab20-11130d5cd553-console-oauth-config\") pod \"console-6699bc67c9-zwmdh\" (UID: \"4a62d81e-42fa-4604-ab20-11130d5cd553\") " pod="openshift-console/console-6699bc67c9-zwmdh" Jul 06 18:20:40.329911 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.329863 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6pgs\" (UniqueName: \"kubernetes.io/projected/4a62d81e-42fa-4604-ab20-11130d5cd553-kube-api-access-f6pgs\") pod \"console-6699bc67c9-zwmdh\" (UID: \"4a62d81e-42fa-4604-ab20-11130d5cd553\") " pod="openshift-console/console-6699bc67c9-zwmdh" Jul 06 18:20:40.507681 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:40.507637 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6699bc67c9-zwmdh" Jul 06 18:20:41.876937 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:41.876902 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jqslw"] Jul 06 18:20:41.904004 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:41.903931 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6699bc67c9-zwmdh"] Jul 06 18:20:41.909074 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:20:41.909037 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a62d81e_42fa_4604_ab20_11130d5cd553.slice/crio-e3cb790038d33e39115dd49832644b5dfddf6452808cab8066f5adbeccb1c3fe WatchSource:0}: Error finding container e3cb790038d33e39115dd49832644b5dfddf6452808cab8066f5adbeccb1c3fe: Status 404 returned error can't find the container with id e3cb790038d33e39115dd49832644b5dfddf6452808cab8066f5adbeccb1c3fe Jul 06 18:20:42.130611 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:42.130521 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rn4lz" event={"ID":"8987aa45-6cac-4458-9d82-3a4e9c85e09e","Type":"ContainerStarted","Data":"193f8aeff6aca86e6eed3465e2303141fd2913056935e3696467d865dccb2f94"} Jul 06 18:20:42.130792 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:42.130624 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rn4lz" event={"ID":"8987aa45-6cac-4458-9d82-3a4e9c85e09e","Type":"ContainerStarted","Data":"a38176267b8b0891b5655c0c0f89ed4e878ad3a378d6936253e5e626ab60b7df"} Jul 06 18:20:42.130792 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:42.130665 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rn4lz" Jul 06 18:20:42.132286 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:42.132260 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4chpk" event={"ID":"01ea6ede-0121-412d-b17c-5d9073fbb347","Type":"ContainerStarted","Data":"774c3a24d328f72829ab4028857027edaf53a092b876a0321aa7d3e06fa369da"} Jul 06 18:20:42.133623 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:42.133603 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-clcjg" event={"ID":"e26e29f4-5957-4076-ab8b-3f741af97a3c","Type":"ContainerStarted","Data":"abee83a881494d1bd6a2385c28235ac9cc4648fc70587deeb7efee1e60c3e5bd"} Jul 06 18:20:42.134783 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:42.134759 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jqslw" event={"ID":"0d0bd084-df2e-4a5e-93ec-4142ba42228a","Type":"ContainerStarted","Data":"90c76a3c7538ac7f409018d87ec86e9c4823606d839c2c751642b8e87b949096"} Jul 06 18:20:42.135879 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:42.135845 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6699bc67c9-zwmdh" event={"ID":"4a62d81e-42fa-4604-ab20-11130d5cd553","Type":"ContainerStarted","Data":"e3cb790038d33e39115dd49832644b5dfddf6452808cab8066f5adbeccb1c3fe"} Jul 06 18:20:42.149764 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:42.149717 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-rn4lz" podStartSLOduration=1.179588222 podStartE2EDuration="6.149704932s" totalImagesPullingTime="4.97011671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:20:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:42.148907381 +0000 UTC m=+58.875336340" watchObservedRunningTime="2026-07-06 18:20:42.149704932 +0000 UTC m=+58.876133890" Jul 06 18:20:42.165010 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:42.164965 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4chpk" podStartSLOduration=1.095930218 podStartE2EDuration="6.164950669s" totalImagesPullingTime="5.069020451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:20:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:42.164384361 +0000 UTC m=+58.890813321" watchObservedRunningTime="2026-07-06 18:20:42.164950669 +0000 UTC m=+58.891379605" Jul 06 18:20:43.142560 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:43.142480 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-clcjg" event={"ID":"e26e29f4-5957-4076-ab8b-3f741af97a3c","Type":"ContainerStarted","Data":"f8a070ef19374bf78450d459c67acd3aaf1e7335512f1e29a6f46ef667faefb4"} Jul 06 18:20:44.148476 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:44.147581 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jqslw" event={"ID":"0d0bd084-df2e-4a5e-93ec-4142ba42228a","Type":"ContainerStarted","Data":"921c2bc0567362904a90b0654d9b1bae24df65ac6d5704b091ab08801515adc3"} Jul 06 18:20:44.148476 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:44.148421 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jqslw" Jul 06 18:20:44.154069 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:44.154037 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jqslw" Jul 06 18:20:44.164004 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:44.163950 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jqslw" podStartSLOduration=3.873746464 podStartE2EDuration="5.163931869s" totalImagesPullingTime="1.290185405s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:20:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:44.16305272 +0000 UTC m=+60.889481668" watchObservedRunningTime="2026-07-06 18:20:44.163931869 +0000 UTC m=+60.890360825" Jul 06 18:20:44.642377 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:44.642345 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:44.674576 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:44.674546 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-kn7k7"] Jul 06 18:20:44.677699 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:44.677676 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kn7k7" Jul 06 18:20:44.682302 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:44.682269 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 18:20:44.682443 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:44.682305 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 18:20:44.682443 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:44.682340 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-rm6jf\"" Jul 06 18:20:44.682443 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:44.682360 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 06 18:20:44.682573 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:44.682275 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 18:20:44.682614 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:44.682468 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 06 18:20:44.686623 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:44.686377 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-kn7k7"] Jul 06 18:20:44.704977 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:44.704942 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:44.758344 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:44.758305 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/43bef29e-0780-4aaf-ad05-d8db529c0ece-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-kn7k7\" (UID: \"43bef29e-0780-4aaf-ad05-d8db529c0ece\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kn7k7" Jul 06 18:20:44.758526 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:44.758393 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96rqp\" (UniqueName: \"kubernetes.io/projected/43bef29e-0780-4aaf-ad05-d8db529c0ece-kube-api-access-96rqp\") pod \"prometheus-operator-567f7b8b7c-kn7k7\" (UID: \"43bef29e-0780-4aaf-ad05-d8db529c0ece\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kn7k7" Jul 06 18:20:44.758526 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:44.758489 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43bef29e-0780-4aaf-ad05-d8db529c0ece-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-kn7k7\" (UID: \"43bef29e-0780-4aaf-ad05-d8db529c0ece\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kn7k7" Jul 06 18:20:44.758664 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:44.758528 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/43bef29e-0780-4aaf-ad05-d8db529c0ece-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-kn7k7\" (UID: \"43bef29e-0780-4aaf-ad05-d8db529c0ece\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kn7k7" Jul 06 18:20:44.859140 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:44.859096 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96rqp\" (UniqueName: \"kubernetes.io/projected/43bef29e-0780-4aaf-ad05-d8db529c0ece-kube-api-access-96rqp\") pod \"prometheus-operator-567f7b8b7c-kn7k7\" (UID: \"43bef29e-0780-4aaf-ad05-d8db529c0ece\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kn7k7" Jul 06 18:20:44.859337 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:44.859155 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43bef29e-0780-4aaf-ad05-d8db529c0ece-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-kn7k7\" (UID: \"43bef29e-0780-4aaf-ad05-d8db529c0ece\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kn7k7" Jul 06 18:20:44.859337 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:44.859198 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/43bef29e-0780-4aaf-ad05-d8db529c0ece-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-kn7k7\" (UID: \"43bef29e-0780-4aaf-ad05-d8db529c0ece\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kn7k7" Jul 06 18:20:44.859337 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:44.859293 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/43bef29e-0780-4aaf-ad05-d8db529c0ece-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-kn7k7\" (UID: \"43bef29e-0780-4aaf-ad05-d8db529c0ece\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kn7k7" Jul 06 18:20:44.859476 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:44.859433 2566 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 06 18:20:44.859517 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:44.859507 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43bef29e-0780-4aaf-ad05-d8db529c0ece-prometheus-operator-tls podName:43bef29e-0780-4aaf-ad05-d8db529c0ece nodeName:}" failed. No retries permitted until 2026-07-06 18:20:45.359473694 +0000 UTC m=+62.085902646 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/43bef29e-0780-4aaf-ad05-d8db529c0ece-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-kn7k7" (UID: "43bef29e-0780-4aaf-ad05-d8db529c0ece") : secret "prometheus-operator-tls" not found Jul 06 18:20:44.859920 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:44.859890 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43bef29e-0780-4aaf-ad05-d8db529c0ece-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-kn7k7\" (UID: \"43bef29e-0780-4aaf-ad05-d8db529c0ece\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kn7k7" Jul 06 18:20:44.862182 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:44.862159 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/43bef29e-0780-4aaf-ad05-d8db529c0ece-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-kn7k7\" (UID: \"43bef29e-0780-4aaf-ad05-d8db529c0ece\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kn7k7" Jul 06 18:20:44.868434 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:44.868412 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96rqp\" (UniqueName: \"kubernetes.io/projected/43bef29e-0780-4aaf-ad05-d8db529c0ece-kube-api-access-96rqp\") pod \"prometheus-operator-567f7b8b7c-kn7k7\" (UID: \"43bef29e-0780-4aaf-ad05-d8db529c0ece\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kn7k7" Jul 06 18:20:45.047606 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:45.047469 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ffzjr" Jul 06 18:20:45.050433 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:45.050401 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:45.191351 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:45.191316 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:45.292408 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:45.292372 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:45.365761 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:45.365660 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/43bef29e-0780-4aaf-ad05-d8db529c0ece-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-kn7k7\" (UID: \"43bef29e-0780-4aaf-ad05-d8db529c0ece\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kn7k7" Jul 06 18:20:45.368818 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:45.368790 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/43bef29e-0780-4aaf-ad05-d8db529c0ece-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-kn7k7\" (UID: \"43bef29e-0780-4aaf-ad05-d8db529c0ece\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kn7k7" Jul 06 18:20:45.592292 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:45.592247 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kn7k7" Jul 06 18:20:47.124685 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:47.124572 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-kn7k7"] Jul 06 18:20:47.127713 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:20:47.127683 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43bef29e_0780_4aaf_ad05_d8db529c0ece.slice/crio-3a8c4f8fca0324682dbb3934808bac45d9030b8a93dc943da122e001ae483edb WatchSource:0}: Error finding container 3a8c4f8fca0324682dbb3934808bac45d9030b8a93dc943da122e001ae483edb: Status 404 returned error can't find the container with id 3a8c4f8fca0324682dbb3934808bac45d9030b8a93dc943da122e001ae483edb Jul 06 18:20:47.161317 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:47.161279 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-clcjg" event={"ID":"e26e29f4-5957-4076-ab8b-3f741af97a3c","Type":"ContainerStarted","Data":"aec67876f1710400400dbf56ea04052336d6aa3f7856bad690a7351775ea4082"} Jul 06 18:20:47.162822 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:47.162792 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6699bc67c9-zwmdh" event={"ID":"4a62d81e-42fa-4604-ab20-11130d5cd553","Type":"ContainerStarted","Data":"b05f8f864308ab3bd8e408a072e52acb6d3a0df3f52e89b2aa93695f12efdcac"} Jul 06 18:20:47.164000 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:47.163978 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kn7k7" event={"ID":"43bef29e-0780-4aaf-ad05-d8db529c0ece","Type":"ContainerStarted","Data":"3a8c4f8fca0324682dbb3934808bac45d9030b8a93dc943da122e001ae483edb"} Jul 06 18:20:47.181110 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:47.181061 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-clcjg" podStartSLOduration=0.986934634 podStartE2EDuration="11.181047646s" totalImagesPullingTime="10.194113012s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:20:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:47.179221236 +0000 UTC m=+63.905650195" watchObservedRunningTime="2026-07-06 18:20:47.181047646 +0000 UTC m=+63.907476603" Jul 06 18:20:47.197368 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:47.197318 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6699bc67c9-zwmdh" podStartSLOduration=2.115593017 podStartE2EDuration="7.19730196s" totalImagesPullingTime="5.081708943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:20:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:47.196355781 +0000 UTC m=+63.922784738" watchObservedRunningTime="2026-07-06 18:20:47.19730196 +0000 UTC m=+63.923730974" Jul 06 18:20:47.969801 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:47.969764 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-664b87676-6zq6q"] Jul 06 18:20:47.971671 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:47.971651 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-664b87676-6zq6q" Jul 06 18:20:47.981233 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:47.981199 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 18:20:47.984365 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:47.984333 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-664b87676-6zq6q"] Jul 06 18:20:47.988436 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:47.988410 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6g82\" (UniqueName: \"kubernetes.io/projected/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-kube-api-access-n6g82\") pod \"console-664b87676-6zq6q\" (UID: \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\") " pod="openshift-console/console-664b87676-6zq6q" Jul 06 18:20:47.988578 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:47.988466 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-trusted-ca-bundle\") pod \"console-664b87676-6zq6q\" (UID: \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\") " pod="openshift-console/console-664b87676-6zq6q" Jul 06 18:20:47.988578 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:47.988543 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-console-config\") pod \"console-664b87676-6zq6q\" (UID: \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\") " pod="openshift-console/console-664b87676-6zq6q" Jul 06 18:20:47.988578 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:47.988572 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-console-serving-cert\") pod \"console-664b87676-6zq6q\" (UID: \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\") " pod="openshift-console/console-664b87676-6zq6q" Jul 06 18:20:47.988730 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:47.988610 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-console-oauth-config\") pod \"console-664b87676-6zq6q\" (UID: \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\") " pod="openshift-console/console-664b87676-6zq6q" Jul 06 18:20:47.988730 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:47.988629 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-service-ca\") pod \"console-664b87676-6zq6q\" (UID: \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\") " pod="openshift-console/console-664b87676-6zq6q" Jul 06 18:20:47.988730 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:47.988649 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-oauth-serving-cert\") pod \"console-664b87676-6zq6q\" (UID: \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\") " pod="openshift-console/console-664b87676-6zq6q" Jul 06 18:20:48.089197 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.089162 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-console-config\") pod \"console-664b87676-6zq6q\" (UID: \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\") " pod="openshift-console/console-664b87676-6zq6q" Jul 06 18:20:48.089357 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.089212 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-console-serving-cert\") pod \"console-664b87676-6zq6q\" (UID: \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\") " pod="openshift-console/console-664b87676-6zq6q" Jul 06 18:20:48.089411 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.089378 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-console-oauth-config\") pod \"console-664b87676-6zq6q\" (UID: \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\") " pod="openshift-console/console-664b87676-6zq6q" Jul 06 18:20:48.089458 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.089424 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-service-ca\") pod \"console-664b87676-6zq6q\" (UID: \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\") " pod="openshift-console/console-664b87676-6zq6q" Jul 06 18:20:48.089519 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.089462 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-oauth-serving-cert\") pod \"console-664b87676-6zq6q\" (UID: \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\") " pod="openshift-console/console-664b87676-6zq6q" Jul 06 18:20:48.089572 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.089516 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6g82\" (UniqueName: \"kubernetes.io/projected/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-kube-api-access-n6g82\") pod \"console-664b87676-6zq6q\" (UID: \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\") " pod="openshift-console/console-664b87676-6zq6q" Jul 06 18:20:48.089619 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.089568 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-trusted-ca-bundle\") pod \"console-664b87676-6zq6q\" (UID: \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\") " pod="openshift-console/console-664b87676-6zq6q" Jul 06 18:20:48.090135 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.089912 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-console-config\") pod \"console-664b87676-6zq6q\" (UID: \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\") " pod="openshift-console/console-664b87676-6zq6q" Jul 06 18:20:48.090270 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.090250 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-oauth-serving-cert\") pod \"console-664b87676-6zq6q\" (UID: \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\") " pod="openshift-console/console-664b87676-6zq6q" Jul 06 18:20:48.090523 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.090494 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-trusted-ca-bundle\") pod \"console-664b87676-6zq6q\" (UID: \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\") " pod="openshift-console/console-664b87676-6zq6q" Jul 06 18:20:48.090894 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.090852 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-service-ca\") pod \"console-664b87676-6zq6q\" (UID: \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\") " pod="openshift-console/console-664b87676-6zq6q" Jul 06 18:20:48.092053 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.092029 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-console-oauth-config\") pod \"console-664b87676-6zq6q\" (UID: \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\") " pod="openshift-console/console-664b87676-6zq6q" Jul 06 18:20:48.092152 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.092138 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-console-serving-cert\") pod \"console-664b87676-6zq6q\" (UID: \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\") " pod="openshift-console/console-664b87676-6zq6q" Jul 06 18:20:48.100173 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.100150 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6g82\" (UniqueName: \"kubernetes.io/projected/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-kube-api-access-n6g82\") pod \"console-664b87676-6zq6q\" (UID: \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\") " pod="openshift-console/console-664b87676-6zq6q" Jul 06 18:20:48.288689 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.288607 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-664b87676-6zq6q" Jul 06 18:20:48.492118 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.492079 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3157c562-b33e-4c9c-b724-cdac9220bbb0-original-pull-secret\") pod \"global-pull-secret-syncer-5hbx6\" (UID: \"3157c562-b33e-4c9c-b724-cdac9220bbb0\") " pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:20:48.495424 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.495400 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 18:20:48.505752 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.505720 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3157c562-b33e-4c9c-b724-cdac9220bbb0-original-pull-secret\") pod \"global-pull-secret-syncer-5hbx6\" (UID: \"3157c562-b33e-4c9c-b724-cdac9220bbb0\") " pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:20:48.593486 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.593444 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2c962e6-4904-4e1b-89f8-b6acb40f6a80-metrics-certs\") pod \"network-metrics-daemon-4gmr8\" (UID: \"f2c962e6-4904-4e1b-89f8-b6acb40f6a80\") " pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:20:48.593602 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.593504 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvx8j\" (UniqueName: \"kubernetes.io/projected/f142192b-8eb7-419d-bfc0-a97171b652b7-kube-api-access-fvx8j\") pod \"network-check-target-n4j4l\" (UID: \"f142192b-8eb7-419d-bfc0-a97171b652b7\") " pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:20:48.596700 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.596681 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 18:20:48.596700 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.596694 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 18:20:48.606903 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.606882 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 18:20:48.607419 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.607388 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2c962e6-4904-4e1b-89f8-b6acb40f6a80-metrics-certs\") pod \"network-metrics-daemon-4gmr8\" (UID: \"f2c962e6-4904-4e1b-89f8-b6acb40f6a80\") " pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:20:48.616711 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.616678 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvx8j\" (UniqueName: \"kubernetes.io/projected/f142192b-8eb7-419d-bfc0-a97171b652b7-kube-api-access-fvx8j\") pod \"network-check-target-n4j4l\" (UID: \"f142192b-8eb7-419d-bfc0-a97171b652b7\") " pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:20:48.659564 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.659534 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-664b87676-6zq6q"] Jul 06 18:20:48.663281 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:20:48.663209 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe5705f0_bde0_4b0f_a699_5f2b6b3d8018.slice/crio-ecfe1c0e132e3f7d8e1972d556523814e29be64b74533ffa1a03c74242bb8bf4 WatchSource:0}: Error finding container ecfe1c0e132e3f7d8e1972d556523814e29be64b74533ffa1a03c74242bb8bf4: Status 404 returned error can't find the container with id ecfe1c0e132e3f7d8e1972d556523814e29be64b74533ffa1a03c74242bb8bf4 Jul 06 18:20:48.758655 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.758612 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mpmrs\"" Jul 06 18:20:48.765785 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.765758 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:20:48.765954 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.765805 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rgvgt\"" Jul 06 18:20:48.770251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.770233 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5hbx6" Jul 06 18:20:48.773763 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.773734 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4gmr8" Jul 06 18:20:48.876719 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.876617 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:48.936342 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:48.936315 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n4j4l"] Jul 06 18:20:48.939119 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:20:48.939089 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf142192b_8eb7_419d_bfc0_a97171b652b7.slice/crio-aab9605562544431f124505ead9f5ece4610c07ebccd6ec7ecdb7a0283f7ce77 WatchSource:0}: Error finding container aab9605562544431f124505ead9f5ece4610c07ebccd6ec7ecdb7a0283f7ce77: Status 404 returned error can't find the container with id aab9605562544431f124505ead9f5ece4610c07ebccd6ec7ecdb7a0283f7ce77 Jul 06 18:20:49.160399 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:49.160375 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4gmr8"] Jul 06 18:20:49.161289 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:49.161211 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5hbx6"] Jul 06 18:20:49.162968 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:20:49.162940 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3157c562_b33e_4c9c_b724_cdac9220bbb0.slice/crio-908a418b12712531c620e058a87b8cf43f03f88f6a5143c324c53f3db0a13168 WatchSource:0}: Error finding container 908a418b12712531c620e058a87b8cf43f03f88f6a5143c324c53f3db0a13168: Status 404 returned error can't find the container with id 908a418b12712531c620e058a87b8cf43f03f88f6a5143c324c53f3db0a13168 Jul 06 18:20:49.163656 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:20:49.163633 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2c962e6_4904_4e1b_89f8_b6acb40f6a80.slice/crio-9bbfb532088e91b3c82fd6610fa7b8d1fb29845c0084bdd30d7251983af39ad7 WatchSource:0}: Error finding container 9bbfb532088e91b3c82fd6610fa7b8d1fb29845c0084bdd30d7251983af39ad7: Status 404 returned error can't find the container with id 9bbfb532088e91b3c82fd6610fa7b8d1fb29845c0084bdd30d7251983af39ad7 Jul 06 18:20:49.172463 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:49.172438 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kn7k7" event={"ID":"43bef29e-0780-4aaf-ad05-d8db529c0ece","Type":"ContainerStarted","Data":"74020f6657304157d47675b063e192dc21bcb23829cca1e238ac56295fcf3c7c"} Jul 06 18:20:49.172556 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:49.172490 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kn7k7" event={"ID":"43bef29e-0780-4aaf-ad05-d8db529c0ece","Type":"ContainerStarted","Data":"dcdc51aedf1bdf463d1f8eaba5dc6a36ba8acb6cd3edff9fb5eaf1a2c029e985"} Jul 06 18:20:49.173619 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:49.173598 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5hbx6" event={"ID":"3157c562-b33e-4c9c-b724-cdac9220bbb0","Type":"ContainerStarted","Data":"908a418b12712531c620e058a87b8cf43f03f88f6a5143c324c53f3db0a13168"} Jul 06 18:20:49.174732 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:49.174699 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4gmr8" event={"ID":"f2c962e6-4904-4e1b-89f8-b6acb40f6a80","Type":"ContainerStarted","Data":"9bbfb532088e91b3c82fd6610fa7b8d1fb29845c0084bdd30d7251983af39ad7"} Jul 06 18:20:49.176272 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:49.176248 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-664b87676-6zq6q" event={"ID":"be5705f0-bde0-4b0f-a699-5f2b6b3d8018","Type":"ContainerStarted","Data":"dd230413a7775ca5e2b8e38f4b7177015f2e29aa740c184278fc4be429b6098d"} Jul 06 18:20:49.176344 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:49.176287 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-664b87676-6zq6q" event={"ID":"be5705f0-bde0-4b0f-a699-5f2b6b3d8018","Type":"ContainerStarted","Data":"ecfe1c0e132e3f7d8e1972d556523814e29be64b74533ffa1a03c74242bb8bf4"} Jul 06 18:20:49.177739 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:49.177721 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n4j4l" event={"ID":"f142192b-8eb7-419d-bfc0-a97171b652b7","Type":"ContainerStarted","Data":"aab9605562544431f124505ead9f5ece4610c07ebccd6ec7ecdb7a0283f7ce77"} Jul 06 18:20:49.190387 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:49.190341 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kn7k7" podStartSLOduration=3.750385114 podStartE2EDuration="5.190326899s" totalImagesPullingTime="1.439941785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:20:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:49.189407451 +0000 UTC m=+65.915836450" watchObservedRunningTime="2026-07-06 18:20:49.190326899 +0000 UTC m=+65.916755857" Jul 06 18:20:49.206944 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:49.206898 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-664b87676-6zq6q" podStartSLOduration=2.206884342 podStartE2EDuration="2.206884342s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:20:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:49.205727973 +0000 UTC m=+65.932156932" watchObservedRunningTime="2026-07-06 18:20:49.206884342 +0000 UTC m=+65.933313290" Jul 06 18:20:49.372359 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:49.372321 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:50.508242 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:50.508144 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6699bc67c9-zwmdh" Jul 06 18:20:50.508668 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:50.508249 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6699bc67c9-zwmdh" Jul 06 18:20:50.510024 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:50.509697 2566 patch_prober.go:28] interesting pod/console-6699bc67c9-zwmdh container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.11:8443/health\": dial tcp 10.134.0.11:8443: connect: connection refused" start-of-body= Jul 06 18:20:50.510024 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:50.509750 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-6699bc67c9-zwmdh" podUID="4a62d81e-42fa-4604-ab20-11130d5cd553" containerName="console" probeResult="failure" output="Get \"https://10.134.0.11:8443/health\": dial tcp 10.134.0.11:8443: connect: connection refused" Jul 06 18:20:50.781698 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:50.781600 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:50.926852 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:50.926806 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:51.021041 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.021012 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:51.025089 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.025052 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-qj66f"] Jul 06 18:20:51.027648 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.027625 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qj66f" Jul 06 18:20:51.030693 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.030671 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 06 18:20:51.030819 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.030678 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-rrf65\"" Jul 06 18:20:51.030924 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.030903 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 06 18:20:51.042818 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.042788 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-qj66f"] Jul 06 18:20:51.043690 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.043666 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-sxg68"] Jul 06 18:20:51.046233 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.046215 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-sxg68" Jul 06 18:20:51.049359 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.049153 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 06 18:20:51.049359 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.049228 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 06 18:20:51.049359 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.049277 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 06 18:20:51.049359 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.049318 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-6nkhv\"" Jul 06 18:20:51.060209 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.060184 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-sxg68"] Jul 06 18:20:51.063927 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.063902 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-k292x"] Jul 06 18:20:51.066647 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.066619 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:51.069669 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.069645 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 18:20:51.069769 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.069645 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 18:20:51.069769 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.069645 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5f786\"" Jul 06 18:20:51.069909 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.069653 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 18:20:51.093752 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.093721 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:51.115129 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.115091 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ba103d37-a01d-4d1b-afab-5a92105dd1cc-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-sxg68\" (UID: \"ba103d37-a01d-4d1b-afab-5a92105dd1cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-sxg68" Jul 06 18:20:51.115129 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.115129 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/63698879-bca0-4fb4-944b-3151be7fbc11-node-exporter-accelerators-collector-config\") pod \"node-exporter-k292x\" (UID: \"63698879-bca0-4fb4-944b-3151be7fbc11\") " pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:51.115337 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.115160 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/63698879-bca0-4fb4-944b-3151be7fbc11-node-exporter-wtmp\") pod \"node-exporter-k292x\" (UID: \"63698879-bca0-4fb4-944b-3151be7fbc11\") " pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:51.115337 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.115211 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba103d37-a01d-4d1b-afab-5a92105dd1cc-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-sxg68\" (UID: \"ba103d37-a01d-4d1b-afab-5a92105dd1cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-sxg68" Jul 06 18:20:51.115337 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.115242 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/63698879-bca0-4fb4-944b-3151be7fbc11-root\") pod \"node-exporter-k292x\" (UID: \"63698879-bca0-4fb4-944b-3151be7fbc11\") " pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:51.115337 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.115326 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkn6q\" (UniqueName: \"kubernetes.io/projected/ba103d37-a01d-4d1b-afab-5a92105dd1cc-kube-api-access-gkn6q\") pod \"kube-state-metrics-7c9dbb8b7d-sxg68\" (UID: \"ba103d37-a01d-4d1b-afab-5a92105dd1cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-sxg68" Jul 06 18:20:51.115481 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.115352 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba103d37-a01d-4d1b-afab-5a92105dd1cc-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-sxg68\" (UID: \"ba103d37-a01d-4d1b-afab-5a92105dd1cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-sxg68" Jul 06 18:20:51.115481 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.115375 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63698879-bca0-4fb4-944b-3151be7fbc11-sys\") pod \"node-exporter-k292x\" (UID: \"63698879-bca0-4fb4-944b-3151be7fbc11\") " pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:51.115481 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.115390 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/63698879-bca0-4fb4-944b-3151be7fbc11-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k292x\" (UID: \"63698879-bca0-4fb4-944b-3151be7fbc11\") " pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:51.115481 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.115406 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ba103d37-a01d-4d1b-afab-5a92105dd1cc-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-sxg68\" (UID: \"ba103d37-a01d-4d1b-afab-5a92105dd1cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-sxg68" Jul 06 18:20:51.115481 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.115434 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ba103d37-a01d-4d1b-afab-5a92105dd1cc-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-sxg68\" (UID: \"ba103d37-a01d-4d1b-afab-5a92105dd1cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-sxg68" Jul 06 18:20:51.115640 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.115490 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/63698879-bca0-4fb4-944b-3151be7fbc11-node-exporter-textfile\") pod \"node-exporter-k292x\" (UID: \"63698879-bca0-4fb4-944b-3151be7fbc11\") " pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:51.115640 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.115508 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9bda0943-9d88-4367-be22-9ec970388f56-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-qj66f\" (UID: \"9bda0943-9d88-4367-be22-9ec970388f56\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qj66f" Jul 06 18:20:51.115640 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.115526 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9bda0943-9d88-4367-be22-9ec970388f56-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-qj66f\" (UID: \"9bda0943-9d88-4367-be22-9ec970388f56\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qj66f" Jul 06 18:20:51.115640 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.115558 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9bda0943-9d88-4367-be22-9ec970388f56-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-qj66f\" (UID: \"9bda0943-9d88-4367-be22-9ec970388f56\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qj66f" Jul 06 18:20:51.115640 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.115574 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2bzb\" (UniqueName: \"kubernetes.io/projected/9bda0943-9d88-4367-be22-9ec970388f56-kube-api-access-p2bzb\") pod \"openshift-state-metrics-54cf96dfcf-qj66f\" (UID: \"9bda0943-9d88-4367-be22-9ec970388f56\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qj66f" Jul 06 18:20:51.115640 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.115600 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/63698879-bca0-4fb4-944b-3151be7fbc11-node-exporter-tls\") pod \"node-exporter-k292x\" (UID: \"63698879-bca0-4fb4-944b-3151be7fbc11\") " pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:51.115640 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.115619 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63698879-bca0-4fb4-944b-3151be7fbc11-metrics-client-ca\") pod \"node-exporter-k292x\" (UID: \"63698879-bca0-4fb4-944b-3151be7fbc11\") " pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:51.115848 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.115646 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ck54n\" (UniqueName: \"kubernetes.io/projected/63698879-bca0-4fb4-944b-3151be7fbc11-kube-api-access-ck54n\") pod \"node-exporter-k292x\" (UID: \"63698879-bca0-4fb4-944b-3151be7fbc11\") " pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:51.163257 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.163224 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:51.188272 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.188233 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4gmr8" event={"ID":"f2c962e6-4904-4e1b-89f8-b6acb40f6a80","Type":"ContainerStarted","Data":"8449c0b46366f9cfa214464796dd2480fc003044098f9a212b89376c7b8081ba"} Jul 06 18:20:51.188419 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.188292 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4gmr8" event={"ID":"f2c962e6-4904-4e1b-89f8-b6acb40f6a80","Type":"ContainerStarted","Data":"90cad05a28693b7f1607e99f4c8c6ce6ab1cc155ea71169d6be483d9e432667e"} Jul 06 18:20:51.205214 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.205159 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4gmr8" podStartSLOduration=66.168014252 podStartE2EDuration="1m7.205140058s" totalImagesPullingTime="1.037125806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:19:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:51.20507336 +0000 UTC m=+67.931502319" watchObservedRunningTime="2026-07-06 18:20:51.205140058 +0000 UTC m=+67.931569017" Jul 06 18:20:51.216129 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.216087 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/63698879-bca0-4fb4-944b-3151be7fbc11-node-exporter-wtmp\") pod \"node-exporter-k292x\" (UID: \"63698879-bca0-4fb4-944b-3151be7fbc11\") " pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:51.216310 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.216138 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba103d37-a01d-4d1b-afab-5a92105dd1cc-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-sxg68\" (UID: \"ba103d37-a01d-4d1b-afab-5a92105dd1cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-sxg68" Jul 06 18:20:51.216310 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.216167 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/63698879-bca0-4fb4-944b-3151be7fbc11-root\") pod \"node-exporter-k292x\" (UID: \"63698879-bca0-4fb4-944b-3151be7fbc11\") " pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:51.216310 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.216215 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkn6q\" (UniqueName: \"kubernetes.io/projected/ba103d37-a01d-4d1b-afab-5a92105dd1cc-kube-api-access-gkn6q\") pod \"kube-state-metrics-7c9dbb8b7d-sxg68\" (UID: \"ba103d37-a01d-4d1b-afab-5a92105dd1cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-sxg68" Jul 06 18:20:51.216310 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.216245 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba103d37-a01d-4d1b-afab-5a92105dd1cc-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-sxg68\" (UID: \"ba103d37-a01d-4d1b-afab-5a92105dd1cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-sxg68" Jul 06 18:20:51.216310 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.216273 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63698879-bca0-4fb4-944b-3151be7fbc11-sys\") pod \"node-exporter-k292x\" (UID: \"63698879-bca0-4fb4-944b-3151be7fbc11\") " pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:51.216310 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.216297 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/63698879-bca0-4fb4-944b-3151be7fbc11-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k292x\" (UID: \"63698879-bca0-4fb4-944b-3151be7fbc11\") " pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:51.216608 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.216321 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ba103d37-a01d-4d1b-afab-5a92105dd1cc-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-sxg68\" (UID: \"ba103d37-a01d-4d1b-afab-5a92105dd1cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-sxg68" Jul 06 18:20:51.216608 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.216347 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ba103d37-a01d-4d1b-afab-5a92105dd1cc-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-sxg68\" (UID: \"ba103d37-a01d-4d1b-afab-5a92105dd1cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-sxg68" Jul 06 18:20:51.216608 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.216384 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/63698879-bca0-4fb4-944b-3151be7fbc11-node-exporter-textfile\") pod \"node-exporter-k292x\" (UID: \"63698879-bca0-4fb4-944b-3151be7fbc11\") " pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:51.216608 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.216413 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9bda0943-9d88-4367-be22-9ec970388f56-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-qj66f\" (UID: \"9bda0943-9d88-4367-be22-9ec970388f56\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qj66f" Jul 06 18:20:51.216608 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.216439 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9bda0943-9d88-4367-be22-9ec970388f56-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-qj66f\" (UID: \"9bda0943-9d88-4367-be22-9ec970388f56\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qj66f" Jul 06 18:20:51.216608 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.216482 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9bda0943-9d88-4367-be22-9ec970388f56-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-qj66f\" (UID: \"9bda0943-9d88-4367-be22-9ec970388f56\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qj66f" Jul 06 18:20:51.216608 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.216508 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2bzb\" (UniqueName: \"kubernetes.io/projected/9bda0943-9d88-4367-be22-9ec970388f56-kube-api-access-p2bzb\") pod \"openshift-state-metrics-54cf96dfcf-qj66f\" (UID: \"9bda0943-9d88-4367-be22-9ec970388f56\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qj66f" Jul 06 18:20:51.216608 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.216542 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/63698879-bca0-4fb4-944b-3151be7fbc11-node-exporter-tls\") pod \"node-exporter-k292x\" (UID: \"63698879-bca0-4fb4-944b-3151be7fbc11\") " pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:51.216608 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.216569 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63698879-bca0-4fb4-944b-3151be7fbc11-metrics-client-ca\") pod \"node-exporter-k292x\" (UID: \"63698879-bca0-4fb4-944b-3151be7fbc11\") " pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:51.216608 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.216599 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ck54n\" (UniqueName: \"kubernetes.io/projected/63698879-bca0-4fb4-944b-3151be7fbc11-kube-api-access-ck54n\") pod \"node-exporter-k292x\" (UID: \"63698879-bca0-4fb4-944b-3151be7fbc11\") " pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:51.216954 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.216630 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ba103d37-a01d-4d1b-afab-5a92105dd1cc-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-sxg68\" (UID: \"ba103d37-a01d-4d1b-afab-5a92105dd1cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-sxg68" Jul 06 18:20:51.216954 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.216659 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/63698879-bca0-4fb4-944b-3151be7fbc11-node-exporter-accelerators-collector-config\") pod \"node-exporter-k292x\" (UID: \"63698879-bca0-4fb4-944b-3151be7fbc11\") " pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:51.217304 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.217281 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/63698879-bca0-4fb4-944b-3151be7fbc11-node-exporter-accelerators-collector-config\") pod \"node-exporter-k292x\" (UID: \"63698879-bca0-4fb4-944b-3151be7fbc11\") " pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:51.217919 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.217510 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/63698879-bca0-4fb4-944b-3151be7fbc11-node-exporter-textfile\") pod \"node-exporter-k292x\" (UID: \"63698879-bca0-4fb4-944b-3151be7fbc11\") " pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:51.217919 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.217577 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/63698879-bca0-4fb4-944b-3151be7fbc11-node-exporter-wtmp\") pod \"node-exporter-k292x\" (UID: \"63698879-bca0-4fb4-944b-3151be7fbc11\") " pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:51.218186 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.218165 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9bda0943-9d88-4367-be22-9ec970388f56-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-qj66f\" (UID: \"9bda0943-9d88-4367-be22-9ec970388f56\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qj66f" Jul 06 18:20:51.218266 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.218168 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba103d37-a01d-4d1b-afab-5a92105dd1cc-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-sxg68\" (UID: \"ba103d37-a01d-4d1b-afab-5a92105dd1cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-sxg68" Jul 06 18:20:51.218266 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.218212 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/63698879-bca0-4fb4-944b-3151be7fbc11-root\") pod \"node-exporter-k292x\" (UID: \"63698879-bca0-4fb4-944b-3151be7fbc11\") " pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:51.218367 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:51.218274 2566 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 06 18:20:51.218367 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:51.218339 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bda0943-9d88-4367-be22-9ec970388f56-openshift-state-metrics-tls podName:9bda0943-9d88-4367-be22-9ec970388f56 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:51.718321276 +0000 UTC m=+68.444750218 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/9bda0943-9d88-4367-be22-9ec970388f56-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-qj66f" (UID: "9bda0943-9d88-4367-be22-9ec970388f56") : secret "openshift-state-metrics-tls" not found Jul 06 18:20:51.219233 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.219198 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ba103d37-a01d-4d1b-afab-5a92105dd1cc-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-sxg68\" (UID: \"ba103d37-a01d-4d1b-afab-5a92105dd1cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-sxg68" Jul 06 18:20:51.219340 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.219269 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63698879-bca0-4fb4-944b-3151be7fbc11-sys\") pod \"node-exporter-k292x\" (UID: \"63698879-bca0-4fb4-944b-3151be7fbc11\") " pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:51.219473 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.219447 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63698879-bca0-4fb4-944b-3151be7fbc11-metrics-client-ca\") pod \"node-exporter-k292x\" (UID: \"63698879-bca0-4fb4-944b-3151be7fbc11\") " pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:51.219534 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:51.219457 2566 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 06 18:20:51.219590 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:51.219548 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63698879-bca0-4fb4-944b-3151be7fbc11-node-exporter-tls podName:63698879-bca0-4fb4-944b-3151be7fbc11 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:51.719528677 +0000 UTC m=+68.445957631 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/63698879-bca0-4fb4-944b-3151be7fbc11-node-exporter-tls") pod "node-exporter-k292x" (UID: "63698879-bca0-4fb4-944b-3151be7fbc11") : secret "node-exporter-tls" not found Jul 06 18:20:51.219654 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:51.219586 2566 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 06 18:20:51.219808 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:51.219711 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba103d37-a01d-4d1b-afab-5a92105dd1cc-kube-state-metrics-tls podName:ba103d37-a01d-4d1b-afab-5a92105dd1cc nodeName:}" failed. No retries permitted until 2026-07-06 18:20:51.71967966 +0000 UTC m=+68.446108598 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/ba103d37-a01d-4d1b-afab-5a92105dd1cc-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-sxg68" (UID: "ba103d37-a01d-4d1b-afab-5a92105dd1cc") : secret "kube-state-metrics-tls" not found Jul 06 18:20:51.220065 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.220041 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ba103d37-a01d-4d1b-afab-5a92105dd1cc-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-sxg68\" (UID: \"ba103d37-a01d-4d1b-afab-5a92105dd1cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-sxg68" Jul 06 18:20:51.220553 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.220526 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ba103d37-a01d-4d1b-afab-5a92105dd1cc-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-sxg68\" (UID: \"ba103d37-a01d-4d1b-afab-5a92105dd1cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-sxg68" Jul 06 18:20:51.221677 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.221653 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/63698879-bca0-4fb4-944b-3151be7fbc11-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k292x\" (UID: \"63698879-bca0-4fb4-944b-3151be7fbc11\") " pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:51.223104 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.223083 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9bda0943-9d88-4367-be22-9ec970388f56-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-qj66f\" (UID: \"9bda0943-9d88-4367-be22-9ec970388f56\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qj66f" Jul 06 18:20:51.229110 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.229090 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkn6q\" (UniqueName: \"kubernetes.io/projected/ba103d37-a01d-4d1b-afab-5a92105dd1cc-kube-api-access-gkn6q\") pod \"kube-state-metrics-7c9dbb8b7d-sxg68\" (UID: \"ba103d37-a01d-4d1b-afab-5a92105dd1cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-sxg68" Jul 06 18:20:51.245324 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.245293 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ck54n\" (UniqueName: \"kubernetes.io/projected/63698879-bca0-4fb4-944b-3151be7fbc11-kube-api-access-ck54n\") pod \"node-exporter-k292x\" (UID: \"63698879-bca0-4fb4-944b-3151be7fbc11\") " pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:51.245468 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.245352 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2bzb\" (UniqueName: \"kubernetes.io/projected/9bda0943-9d88-4367-be22-9ec970388f56-kube-api-access-p2bzb\") pod \"openshift-state-metrics-54cf96dfcf-qj66f\" (UID: \"9bda0943-9d88-4367-be22-9ec970388f56\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qj66f" Jul 06 18:20:51.315129 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.315000 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:51.413741 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.413699 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:51.514413 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.514383 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:51.615066 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.614985 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:51.721558 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.721523 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9bda0943-9d88-4367-be22-9ec970388f56-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-qj66f\" (UID: \"9bda0943-9d88-4367-be22-9ec970388f56\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qj66f" Jul 06 18:20:51.721745 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.721586 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/63698879-bca0-4fb4-944b-3151be7fbc11-node-exporter-tls\") pod \"node-exporter-k292x\" (UID: \"63698879-bca0-4fb4-944b-3151be7fbc11\") " pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:51.721745 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.721655 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba103d37-a01d-4d1b-afab-5a92105dd1cc-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-sxg68\" (UID: \"ba103d37-a01d-4d1b-afab-5a92105dd1cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-sxg68" Jul 06 18:20:51.724505 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.724474 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/63698879-bca0-4fb4-944b-3151be7fbc11-node-exporter-tls\") pod \"node-exporter-k292x\" (UID: \"63698879-bca0-4fb4-944b-3151be7fbc11\") " pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:51.724608 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.724566 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9bda0943-9d88-4367-be22-9ec970388f56-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-qj66f\" (UID: \"9bda0943-9d88-4367-be22-9ec970388f56\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qj66f" Jul 06 18:20:51.724652 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.724630 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ba103d37-a01d-4d1b-afab-5a92105dd1cc-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-sxg68\" (UID: \"ba103d37-a01d-4d1b-afab-5a92105dd1cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-sxg68" Jul 06 18:20:51.939151 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.939109 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qj66f" Jul 06 18:20:51.957323 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.957287 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-sxg68" Jul 06 18:20:51.960238 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.960206 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:51.976895 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:51.976491 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-k292x" Jul 06 18:20:52.060525 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.060493 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:52.117215 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.117077 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 18:20:52.120305 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.120282 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.123486 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.123462 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 18:20:52.123717 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.123703 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 18:20:52.124066 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.124050 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 18:20:52.124809 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.124285 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 18:20:52.124809 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.124308 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 18:20:52.124809 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.124327 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 18:20:52.124809 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.124396 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-975lj\"" Jul 06 18:20:52.124809 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.124472 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 18:20:52.124809 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.124477 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 18:20:52.124809 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.124638 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 18:20:52.136310 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.136283 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 18:20:52.137853 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.137814 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:52.145697 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.145671 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rn4lz" Jul 06 18:20:52.225763 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.225678 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dde180b5-8bfa-42de-b9d0-aa713d3c7855-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.225763 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.225720 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4vkk\" (UniqueName: \"kubernetes.io/projected/dde180b5-8bfa-42de-b9d0-aa713d3c7855-kube-api-access-k4vkk\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.225992 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.225780 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde180b5-8bfa-42de-b9d0-aa713d3c7855-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.225992 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.225848 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.225992 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.225903 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dde180b5-8bfa-42de-b9d0-aa713d3c7855-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.225992 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.225943 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-web-config\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.226214 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.225999 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.226214 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.226057 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-config-volume\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.226214 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.226086 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dde180b5-8bfa-42de-b9d0-aa713d3c7855-config-out\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.226214 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.226112 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.226214 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.226150 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.226214 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.226197 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.226508 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.226459 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dde180b5-8bfa-42de-b9d0-aa713d3c7855-tls-assets\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.327736 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.327692 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-config-volume\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.327947 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.327746 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dde180b5-8bfa-42de-b9d0-aa713d3c7855-config-out\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.327947 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.327772 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.327947 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.327808 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.327947 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.327839 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.327947 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.327884 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dde180b5-8bfa-42de-b9d0-aa713d3c7855-tls-assets\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.327947 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.327921 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dde180b5-8bfa-42de-b9d0-aa713d3c7855-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.327947 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.327944 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4vkk\" (UniqueName: \"kubernetes.io/projected/dde180b5-8bfa-42de-b9d0-aa713d3c7855-kube-api-access-k4vkk\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.328316 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.327972 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde180b5-8bfa-42de-b9d0-aa713d3c7855-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.328316 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.328010 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.328316 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.328050 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dde180b5-8bfa-42de-b9d0-aa713d3c7855-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.328316 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.328087 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-web-config\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.328316 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.328136 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.328702 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:52.328390 2566 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 06 18:20:52.328702 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:52.328457 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-secret-alertmanager-main-tls podName:dde180b5-8bfa-42de-b9d0-aa713d3c7855 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:52.828437607 +0000 UTC m=+69.554866543 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "dde180b5-8bfa-42de-b9d0-aa713d3c7855") : secret "alertmanager-main-tls" not found Jul 06 18:20:52.328702 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.328633 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dde180b5-8bfa-42de-b9d0-aa713d3c7855-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.328702 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:20:52.328673 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dde180b5-8bfa-42de-b9d0-aa713d3c7855-alertmanager-trusted-ca-bundle podName:dde180b5-8bfa-42de-b9d0-aa713d3c7855 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:52.82865628 +0000 UTC m=+69.555085241 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/dde180b5-8bfa-42de-b9d0-aa713d3c7855-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "dde180b5-8bfa-42de-b9d0-aa713d3c7855") : configmap references non-existent config key: ca-bundle.crt Jul 06 18:20:52.329498 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.329409 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dde180b5-8bfa-42de-b9d0-aa713d3c7855-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.330990 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.330918 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dde180b5-8bfa-42de-b9d0-aa713d3c7855-config-out\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.331615 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.331567 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.332897 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.332667 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dde180b5-8bfa-42de-b9d0-aa713d3c7855-tls-assets\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.332897 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.332775 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-web-config\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.332897 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.332833 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.333099 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.333072 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-config-volume\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.333099 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.333079 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.333300 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.333280 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.339516 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.339488 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4vkk\" (UniqueName: \"kubernetes.io/projected/dde180b5-8bfa-42de-b9d0-aa713d3c7855-kube-api-access-k4vkk\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.461325 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.461286 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:52.789559 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.789505 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:52.833003 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.832957 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde180b5-8bfa-42de-b9d0-aa713d3c7855-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.833153 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.833012 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.833902 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.833850 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde180b5-8bfa-42de-b9d0-aa713d3c7855-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.835855 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.835827 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:52.870457 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:52.870415 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:53.027799 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.027760 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-85bfd88756-s6985"] Jul 06 18:20:53.034022 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.032754 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" Jul 06 18:20:53.034186 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.034034 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:20:53.037907 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.037885 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-c80f40vno9vig\"" Jul 06 18:20:53.038189 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.037885 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 06 18:20:53.038438 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.037989 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 06 18:20:53.040015 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.039532 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-nx9j6\"" Jul 06 18:20:53.040015 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.039797 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 06 18:20:53.040015 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.039981 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 06 18:20:53.040015 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.039999 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 06 18:20:53.052358 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.052328 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-85bfd88756-s6985"] Jul 06 18:20:53.136361 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.136320 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f34f8e8d-3be3-4770-b3d8-9db846d5dcc6-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-85bfd88756-s6985\" (UID: \"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6\") " pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" Jul 06 18:20:53.136564 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.136384 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f34f8e8d-3be3-4770-b3d8-9db846d5dcc6-secret-grpc-tls\") pod \"thanos-querier-85bfd88756-s6985\" (UID: \"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6\") " pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" Jul 06 18:20:53.136564 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.136422 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f34f8e8d-3be3-4770-b3d8-9db846d5dcc6-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-85bfd88756-s6985\" (UID: \"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6\") " pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" Jul 06 18:20:53.136564 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.136525 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f34f8e8d-3be3-4770-b3d8-9db846d5dcc6-metrics-client-ca\") pod \"thanos-querier-85bfd88756-s6985\" (UID: \"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6\") " pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" Jul 06 18:20:53.136564 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.136552 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f34f8e8d-3be3-4770-b3d8-9db846d5dcc6-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-85bfd88756-s6985\" (UID: \"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6\") " pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" Jul 06 18:20:53.136784 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.136586 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f34f8e8d-3be3-4770-b3d8-9db846d5dcc6-secret-thanos-querier-tls\") pod \"thanos-querier-85bfd88756-s6985\" (UID: \"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6\") " pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" Jul 06 18:20:53.136784 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.136669 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f34f8e8d-3be3-4770-b3d8-9db846d5dcc6-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-85bfd88756-s6985\" (UID: \"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6\") " pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" Jul 06 18:20:53.136784 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.136693 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr7nd\" (UniqueName: \"kubernetes.io/projected/f34f8e8d-3be3-4770-b3d8-9db846d5dcc6-kube-api-access-dr7nd\") pod \"thanos-querier-85bfd88756-s6985\" (UID: \"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6\") " pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" Jul 06 18:20:53.237429 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.237382 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f34f8e8d-3be3-4770-b3d8-9db846d5dcc6-metrics-client-ca\") pod \"thanos-querier-85bfd88756-s6985\" (UID: \"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6\") " pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" Jul 06 18:20:53.237617 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.237439 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f34f8e8d-3be3-4770-b3d8-9db846d5dcc6-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-85bfd88756-s6985\" (UID: \"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6\") " pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" Jul 06 18:20:53.237617 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.237479 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f34f8e8d-3be3-4770-b3d8-9db846d5dcc6-secret-thanos-querier-tls\") pod \"thanos-querier-85bfd88756-s6985\" (UID: \"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6\") " pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" Jul 06 18:20:53.237617 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.237507 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f34f8e8d-3be3-4770-b3d8-9db846d5dcc6-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-85bfd88756-s6985\" (UID: \"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6\") " pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" Jul 06 18:20:53.237617 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.237529 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr7nd\" (UniqueName: \"kubernetes.io/projected/f34f8e8d-3be3-4770-b3d8-9db846d5dcc6-kube-api-access-dr7nd\") pod \"thanos-querier-85bfd88756-s6985\" (UID: \"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6\") " pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" Jul 06 18:20:53.237829 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.237724 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f34f8e8d-3be3-4770-b3d8-9db846d5dcc6-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-85bfd88756-s6985\" (UID: \"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6\") " pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" Jul 06 18:20:53.237829 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.237807 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f34f8e8d-3be3-4770-b3d8-9db846d5dcc6-secret-grpc-tls\") pod \"thanos-querier-85bfd88756-s6985\" (UID: \"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6\") " pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" Jul 06 18:20:53.237955 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.237844 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f34f8e8d-3be3-4770-b3d8-9db846d5dcc6-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-85bfd88756-s6985\" (UID: \"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6\") " pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" Jul 06 18:20:53.238420 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.238257 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f34f8e8d-3be3-4770-b3d8-9db846d5dcc6-metrics-client-ca\") pod \"thanos-querier-85bfd88756-s6985\" (UID: \"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6\") " pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" Jul 06 18:20:53.240633 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.240590 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f34f8e8d-3be3-4770-b3d8-9db846d5dcc6-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-85bfd88756-s6985\" (UID: \"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6\") " pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" Jul 06 18:20:53.240793 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.240748 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f34f8e8d-3be3-4770-b3d8-9db846d5dcc6-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-85bfd88756-s6985\" (UID: \"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6\") " pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" Jul 06 18:20:53.240793 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.240778 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f34f8e8d-3be3-4770-b3d8-9db846d5dcc6-secret-thanos-querier-tls\") pod \"thanos-querier-85bfd88756-s6985\" (UID: \"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6\") " pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" Jul 06 18:20:53.240948 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.240884 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f34f8e8d-3be3-4770-b3d8-9db846d5dcc6-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-85bfd88756-s6985\" (UID: \"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6\") " pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" Jul 06 18:20:53.241288 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.241265 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f34f8e8d-3be3-4770-b3d8-9db846d5dcc6-secret-grpc-tls\") pod \"thanos-querier-85bfd88756-s6985\" (UID: \"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6\") " pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" Jul 06 18:20:53.242130 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.242106 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f34f8e8d-3be3-4770-b3d8-9db846d5dcc6-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-85bfd88756-s6985\" (UID: \"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6\") " pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" Jul 06 18:20:53.248853 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.248826 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr7nd\" (UniqueName: \"kubernetes.io/projected/f34f8e8d-3be3-4770-b3d8-9db846d5dcc6-kube-api-access-dr7nd\") pod \"thanos-querier-85bfd88756-s6985\" (UID: \"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6\") " pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" Jul 06 18:20:53.347131 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.347037 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" Jul 06 18:20:53.987858 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:53.987821 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:54.109177 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:54.109139 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:54.195333 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:54.194931 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:54.203559 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:54.203530 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k292x" event={"ID":"63698879-bca0-4fb4-944b-3151be7fbc11","Type":"ContainerStarted","Data":"5c05842f8f643f1f79e1720a4475409d4a187d2c80ab98d0cbcc062e29548cc4"} Jul 06 18:20:54.205530 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:54.205488 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n4j4l" event={"ID":"f142192b-8eb7-419d-bfc0-a97171b652b7","Type":"ContainerStarted","Data":"9d6431dcd577c9c75204b234959597e68366adda1eddd337977e15dfbdf91797"} Jul 06 18:20:54.205718 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:54.205701 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:20:54.224578 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:20:54.224552 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddde180b5_8bfa_42de_b9d0_aa713d3c7855.slice/crio-8f8a909508a07a56ccbb6ac022e31eafbe527678ef890bc059de273a7d107567 WatchSource:0}: Error finding container 8f8a909508a07a56ccbb6ac022e31eafbe527678ef890bc059de273a7d107567: Status 404 returned error can't find the container with id 8f8a909508a07a56ccbb6ac022e31eafbe527678ef890bc059de273a7d107567 Jul 06 18:20:54.224578 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:54.224585 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 18:20:54.225503 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:54.225450 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-n4j4l" podStartSLOduration=65.150552465 podStartE2EDuration="1m10.225430167s" totalImagesPullingTime="5.074877702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:19:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:54.222988506 +0000 UTC m=+70.949417464" watchObservedRunningTime="2026-07-06 18:20:54.225430167 +0000 UTC m=+70.951859128" Jul 06 18:20:54.289195 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:54.289113 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:54.432107 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:54.432081 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-qj66f"] Jul 06 18:20:54.434327 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:20:54.434299 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bda0943_9d88_4367_be22_9ec970388f56.slice/crio-80837e45cb647aafe9221f0fa9bc95e89920fb16aca51b26fe08a9786c4d42ed WatchSource:0}: Error finding container 80837e45cb647aafe9221f0fa9bc95e89920fb16aca51b26fe08a9786c4d42ed: Status 404 returned error can't find the container with id 80837e45cb647aafe9221f0fa9bc95e89920fb16aca51b26fe08a9786c4d42ed Jul 06 18:20:54.436768 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:54.436216 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-sxg68"] Jul 06 18:20:54.437384 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:54.437360 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-85bfd88756-s6985"] Jul 06 18:20:54.438589 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:20:54.438348 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf34f8e8d_3be3_4770_b3d8_9db846d5dcc6.slice/crio-a2b72b333518290187a983dfebc42abe9952c03c9e7f9df957a957d23d93b924 WatchSource:0}: Error finding container a2b72b333518290187a983dfebc42abe9952c03c9e7f9df957a957d23d93b924: Status 404 returned error can't find the container with id a2b72b333518290187a983dfebc42abe9952c03c9e7f9df957a957d23d93b924 Jul 06 18:20:54.441073 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:20:54.440947 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba103d37_a01d_4d1b_afab_5a92105dd1cc.slice/crio-20b73864ec39f08d132625efb6fdabd61b1b2e4f946f5d59e4a0db82da191eb2 WatchSource:0}: Error finding container 20b73864ec39f08d132625efb6fdabd61b1b2e4f946f5d59e4a0db82da191eb2: Status 404 returned error can't find the container with id 20b73864ec39f08d132625efb6fdabd61b1b2e4f946f5d59e4a0db82da191eb2 Jul 06 18:20:55.131356 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.131312 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:55.213983 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.213713 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5hbx6" event={"ID":"3157c562-b33e-4c9c-b724-cdac9220bbb0","Type":"ContainerStarted","Data":"1b21c409d4b1a91bee51651f477669f1989f11748d3c608cbeef8d7b3880b07e"} Jul 06 18:20:55.216917 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.216714 2566 generic.go:356] "Generic (PLEG): container finished" podID="63698879-bca0-4fb4-944b-3151be7fbc11" containerID="9f49794e0f5b146f6c8ff71056b9cc51a6832fcbc43f8d636d839bbbce97537c" exitCode=0 Jul 06 18:20:55.216917 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.216832 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k292x" event={"ID":"63698879-bca0-4fb4-944b-3151be7fbc11","Type":"ContainerDied","Data":"9f49794e0f5b146f6c8ff71056b9cc51a6832fcbc43f8d636d839bbbce97537c"} Jul 06 18:20:55.219747 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.219720 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" event={"ID":"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6","Type":"ContainerStarted","Data":"a2b72b333518290187a983dfebc42abe9952c03c9e7f9df957a957d23d93b924"} Jul 06 18:20:55.222666 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.222504 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qj66f" event={"ID":"9bda0943-9d88-4367-be22-9ec970388f56","Type":"ContainerStarted","Data":"a7c269da29c51632c4d74afae059232e45c89de6c8b3aa2faf257bc08c7813b2"} Jul 06 18:20:55.222666 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.222562 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qj66f" event={"ID":"9bda0943-9d88-4367-be22-9ec970388f56","Type":"ContainerStarted","Data":"c2d97d269aff334ac66528321a64ef02d339387a8a6c4e0c11e5359621c34997"} Jul 06 18:20:55.222666 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.222590 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qj66f" event={"ID":"9bda0943-9d88-4367-be22-9ec970388f56","Type":"ContainerStarted","Data":"80837e45cb647aafe9221f0fa9bc95e89920fb16aca51b26fe08a9786c4d42ed"} Jul 06 18:20:55.224812 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.224782 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-sxg68" event={"ID":"ba103d37-a01d-4d1b-afab-5a92105dd1cc","Type":"ContainerStarted","Data":"20b73864ec39f08d132625efb6fdabd61b1b2e4f946f5d59e4a0db82da191eb2"} Jul 06 18:20:55.227306 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.227279 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dde180b5-8bfa-42de-b9d0-aa713d3c7855","Type":"ContainerStarted","Data":"8f8a909508a07a56ccbb6ac022e31eafbe527678ef890bc059de273a7d107567"} Jul 06 18:20:55.234392 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.234339 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5hbx6" podStartSLOduration=66.372768435 podStartE2EDuration="1m11.234320901s" totalImagesPullingTime="4.861552466s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:19:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:55.232502723 +0000 UTC m=+71.958931682" watchObservedRunningTime="2026-07-06 18:20:55.234320901 +0000 UTC m=+71.960749853" Jul 06 18:20:55.237275 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.237216 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:55.428185 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.427727 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6754684457-85h2d"] Jul 06 18:20:55.431192 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.431134 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6754684457-85h2d" Jul 06 18:20:55.434118 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.434093 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 06 18:20:55.434254 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.434190 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 06 18:20:55.434653 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.434624 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-45b29\"" Jul 06 18:20:55.435450 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.435081 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ehpkrgv15s4q2\"" Jul 06 18:20:55.435450 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.435294 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 06 18:20:55.435827 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.435661 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 06 18:20:55.446842 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.446005 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6754684457-85h2d"] Jul 06 18:20:55.457532 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.457493 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db2ee9dc-cdec-4fd5-abc4-d049d53b829a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6754684457-85h2d\" (UID: \"db2ee9dc-cdec-4fd5-abc4-d049d53b829a\") " pod="openshift-monitoring/metrics-server-6754684457-85h2d" Jul 06 18:20:55.457675 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.457546 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/db2ee9dc-cdec-4fd5-abc4-d049d53b829a-secret-metrics-server-client-certs\") pod \"metrics-server-6754684457-85h2d\" (UID: \"db2ee9dc-cdec-4fd5-abc4-d049d53b829a\") " pod="openshift-monitoring/metrics-server-6754684457-85h2d" Jul 06 18:20:55.457675 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.457584 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/db2ee9dc-cdec-4fd5-abc4-d049d53b829a-secret-metrics-server-tls\") pod \"metrics-server-6754684457-85h2d\" (UID: \"db2ee9dc-cdec-4fd5-abc4-d049d53b829a\") " pod="openshift-monitoring/metrics-server-6754684457-85h2d" Jul 06 18:20:55.457675 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.457627 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7br9k\" (UniqueName: \"kubernetes.io/projected/db2ee9dc-cdec-4fd5-abc4-d049d53b829a-kube-api-access-7br9k\") pod \"metrics-server-6754684457-85h2d\" (UID: \"db2ee9dc-cdec-4fd5-abc4-d049d53b829a\") " pod="openshift-monitoring/metrics-server-6754684457-85h2d" Jul 06 18:20:55.457829 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.457696 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/db2ee9dc-cdec-4fd5-abc4-d049d53b829a-metrics-server-audit-profiles\") pod \"metrics-server-6754684457-85h2d\" (UID: \"db2ee9dc-cdec-4fd5-abc4-d049d53b829a\") " pod="openshift-monitoring/metrics-server-6754684457-85h2d" Jul 06 18:20:55.457829 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.457752 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/db2ee9dc-cdec-4fd5-abc4-d049d53b829a-audit-log\") pod \"metrics-server-6754684457-85h2d\" (UID: \"db2ee9dc-cdec-4fd5-abc4-d049d53b829a\") " pod="openshift-monitoring/metrics-server-6754684457-85h2d" Jul 06 18:20:55.457829 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.457775 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/db2ee9dc-cdec-4fd5-abc4-d049d53b829a-client-ca-bundle\") pod \"metrics-server-6754684457-85h2d\" (UID: \"db2ee9dc-cdec-4fd5-abc4-d049d53b829a\") " pod="openshift-monitoring/metrics-server-6754684457-85h2d" Jul 06 18:20:55.558759 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.558675 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7br9k\" (UniqueName: \"kubernetes.io/projected/db2ee9dc-cdec-4fd5-abc4-d049d53b829a-kube-api-access-7br9k\") pod \"metrics-server-6754684457-85h2d\" (UID: \"db2ee9dc-cdec-4fd5-abc4-d049d53b829a\") " pod="openshift-monitoring/metrics-server-6754684457-85h2d" Jul 06 18:20:55.558759 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.558756 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/db2ee9dc-cdec-4fd5-abc4-d049d53b829a-metrics-server-audit-profiles\") pod \"metrics-server-6754684457-85h2d\" (UID: \"db2ee9dc-cdec-4fd5-abc4-d049d53b829a\") " pod="openshift-monitoring/metrics-server-6754684457-85h2d" Jul 06 18:20:55.559004 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.558808 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/db2ee9dc-cdec-4fd5-abc4-d049d53b829a-audit-log\") pod \"metrics-server-6754684457-85h2d\" (UID: \"db2ee9dc-cdec-4fd5-abc4-d049d53b829a\") " pod="openshift-monitoring/metrics-server-6754684457-85h2d" Jul 06 18:20:55.559004 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.558839 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/db2ee9dc-cdec-4fd5-abc4-d049d53b829a-client-ca-bundle\") pod \"metrics-server-6754684457-85h2d\" (UID: \"db2ee9dc-cdec-4fd5-abc4-d049d53b829a\") " pod="openshift-monitoring/metrics-server-6754684457-85h2d" Jul 06 18:20:55.559004 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.558900 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db2ee9dc-cdec-4fd5-abc4-d049d53b829a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6754684457-85h2d\" (UID: \"db2ee9dc-cdec-4fd5-abc4-d049d53b829a\") " pod="openshift-monitoring/metrics-server-6754684457-85h2d" Jul 06 18:20:55.559004 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.558935 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/db2ee9dc-cdec-4fd5-abc4-d049d53b829a-secret-metrics-server-client-certs\") pod \"metrics-server-6754684457-85h2d\" (UID: \"db2ee9dc-cdec-4fd5-abc4-d049d53b829a\") " pod="openshift-monitoring/metrics-server-6754684457-85h2d" Jul 06 18:20:55.559004 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.558959 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/db2ee9dc-cdec-4fd5-abc4-d049d53b829a-secret-metrics-server-tls\") pod \"metrics-server-6754684457-85h2d\" (UID: \"db2ee9dc-cdec-4fd5-abc4-d049d53b829a\") " pod="openshift-monitoring/metrics-server-6754684457-85h2d" Jul 06 18:20:55.559771 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.559741 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/db2ee9dc-cdec-4fd5-abc4-d049d53b829a-metrics-server-audit-profiles\") pod \"metrics-server-6754684457-85h2d\" (UID: \"db2ee9dc-cdec-4fd5-abc4-d049d53b829a\") " pod="openshift-monitoring/metrics-server-6754684457-85h2d" Jul 06 18:20:55.560037 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.560015 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/db2ee9dc-cdec-4fd5-abc4-d049d53b829a-audit-log\") pod \"metrics-server-6754684457-85h2d\" (UID: \"db2ee9dc-cdec-4fd5-abc4-d049d53b829a\") " pod="openshift-monitoring/metrics-server-6754684457-85h2d" Jul 06 18:20:55.560112 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.560068 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db2ee9dc-cdec-4fd5-abc4-d049d53b829a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6754684457-85h2d\" (UID: \"db2ee9dc-cdec-4fd5-abc4-d049d53b829a\") " pod="openshift-monitoring/metrics-server-6754684457-85h2d" Jul 06 18:20:55.562072 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.562050 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/db2ee9dc-cdec-4fd5-abc4-d049d53b829a-client-ca-bundle\") pod \"metrics-server-6754684457-85h2d\" (UID: \"db2ee9dc-cdec-4fd5-abc4-d049d53b829a\") " pod="openshift-monitoring/metrics-server-6754684457-85h2d" Jul 06 18:20:55.563286 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.563263 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/db2ee9dc-cdec-4fd5-abc4-d049d53b829a-secret-metrics-server-tls\") pod \"metrics-server-6754684457-85h2d\" (UID: \"db2ee9dc-cdec-4fd5-abc4-d049d53b829a\") " pod="openshift-monitoring/metrics-server-6754684457-85h2d" Jul 06 18:20:55.564370 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.564346 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/db2ee9dc-cdec-4fd5-abc4-d049d53b829a-secret-metrics-server-client-certs\") pod \"metrics-server-6754684457-85h2d\" (UID: \"db2ee9dc-cdec-4fd5-abc4-d049d53b829a\") " pod="openshift-monitoring/metrics-server-6754684457-85h2d" Jul 06 18:20:55.567521 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.567498 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7br9k\" (UniqueName: \"kubernetes.io/projected/db2ee9dc-cdec-4fd5-abc4-d049d53b829a-kube-api-access-7br9k\") pod \"metrics-server-6754684457-85h2d\" (UID: \"db2ee9dc-cdec-4fd5-abc4-d049d53b829a\") " pod="openshift-monitoring/metrics-server-6754684457-85h2d" Jul 06 18:20:55.744741 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.744701 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6754684457-85h2d" Jul 06 18:20:55.878131 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.878060 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-v67rj"] Jul 06 18:20:55.880505 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.880475 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:55.881012 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.880992 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-v67rj" Jul 06 18:20:55.887202 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.887171 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 06 18:20:55.887312 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.887176 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-vp2xr\"" Jul 06 18:20:55.906357 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.906291 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-v67rj"] Jul 06 18:20:55.963314 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.963168 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfsxf\" (UniqueName: \"kubernetes.io/projected/57e592f6-5606-4945-b8e4-87fa68fa0977-kube-api-access-wfsxf\") pod \"monitoring-plugin-d5dcff7cf-v67rj\" (UID: \"57e592f6-5606-4945-b8e4-87fa68fa0977\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-v67rj" Jul 06 18:20:55.963314 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:55.963236 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/57e592f6-5606-4945-b8e4-87fa68fa0977-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-v67rj\" (UID: \"57e592f6-5606-4945-b8e4-87fa68fa0977\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-v67rj" Jul 06 18:20:56.030818 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:56.030770 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:56.064051 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:56.064010 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfsxf\" (UniqueName: \"kubernetes.io/projected/57e592f6-5606-4945-b8e4-87fa68fa0977-kube-api-access-wfsxf\") pod \"monitoring-plugin-d5dcff7cf-v67rj\" (UID: \"57e592f6-5606-4945-b8e4-87fa68fa0977\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-v67rj" Jul 06 18:20:56.064221 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:56.064090 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/57e592f6-5606-4945-b8e4-87fa68fa0977-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-v67rj\" (UID: \"57e592f6-5606-4945-b8e4-87fa68fa0977\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-v67rj" Jul 06 18:20:56.067431 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:56.067402 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/57e592f6-5606-4945-b8e4-87fa68fa0977-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-v67rj\" (UID: \"57e592f6-5606-4945-b8e4-87fa68fa0977\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-v67rj" Jul 06 18:20:56.073964 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:56.073929 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfsxf\" (UniqueName: \"kubernetes.io/projected/57e592f6-5606-4945-b8e4-87fa68fa0977-kube-api-access-wfsxf\") pod \"monitoring-plugin-d5dcff7cf-v67rj\" (UID: \"57e592f6-5606-4945-b8e4-87fa68fa0977\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-v67rj" Jul 06 18:20:56.129030 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:56.128951 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:56.193241 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:56.193210 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-v67rj" Jul 06 18:20:56.420263 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:56.420234 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-v67rj"] Jul 06 18:20:56.431476 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:56.431435 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6754684457-85h2d"] Jul 06 18:20:56.508224 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:56.508192 2566 patch_prober.go:28] interesting pod/image-registry-6cfff669f4-6fmtq container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 06 18:20:56.508339 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:56.508254 2566 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" podUID="056b10b5-54ff-452a-94b2-24134a58f725" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 18:20:57.021817 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:20:57.021786 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb2ee9dc_cdec_4fd5_abc4_d049d53b829a.slice/crio-e5965aa359fde40297665d7680a318e914c7f0c055704fa7a416fa2c90a50878 WatchSource:0}: Error finding container e5965aa359fde40297665d7680a318e914c7f0c055704fa7a416fa2c90a50878: Status 404 returned error can't find the container with id e5965aa359fde40297665d7680a318e914c7f0c055704fa7a416fa2c90a50878 Jul 06 18:20:57.238751 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.238711 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6754684457-85h2d" event={"ID":"db2ee9dc-cdec-4fd5-abc4-d049d53b829a","Type":"ContainerStarted","Data":"e5965aa359fde40297665d7680a318e914c7f0c055704fa7a416fa2c90a50878"} Jul 06 18:20:57.240334 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.240307 2566 generic.go:356] "Generic (PLEG): container finished" podID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerID="140fc9918bb72618333e263c87914003e984c718107f5bf626580029eaffff30" exitCode=0 Jul 06 18:20:57.240658 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.240572 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dde180b5-8bfa-42de-b9d0-aa713d3c7855","Type":"ContainerDied","Data":"140fc9918bb72618333e263c87914003e984c718107f5bf626580029eaffff30"} Jul 06 18:20:57.242522 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.242491 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-v67rj" event={"ID":"57e592f6-5606-4945-b8e4-87fa68fa0977","Type":"ContainerStarted","Data":"d9c0855fd5ff0c78ee286151a9dbe9a635cbe27d7b0a6bb6a75ac8cca558255d"} Jul 06 18:20:57.246927 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.246754 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" event={"ID":"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6","Type":"ContainerStarted","Data":"b1f26c439a1ba91bbdb326c541ac26a6368c65ae13f889499acea5043a748879"} Jul 06 18:20:57.247656 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.247557 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" event={"ID":"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6","Type":"ContainerStarted","Data":"9f0f60d216255c8614b8e0396010e6414aec2d61c6f72852347ead0c014cf47a"} Jul 06 18:20:57.251272 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.251248 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-sxg68" event={"ID":"ba103d37-a01d-4d1b-afab-5a92105dd1cc","Type":"ContainerStarted","Data":"a4fda247224b9ef453ef1042bf808aa40feb40b0c826efb1dca1480b343d27bb"} Jul 06 18:20:57.252270 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.252252 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-sxg68" event={"ID":"ba103d37-a01d-4d1b-afab-5a92105dd1cc","Type":"ContainerStarted","Data":"71730393ff3d72dd9b5d4a006ff8926b167da42c3100ce6305585370a0823e3e"} Jul 06 18:20:57.252597 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.252583 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-sxg68" event={"ID":"ba103d37-a01d-4d1b-afab-5a92105dd1cc","Type":"ContainerStarted","Data":"9b39a3d40c5f56d61baebe19930762e94936e94a2cf2c0d1fd9a2b5efc5e2112"} Jul 06 18:20:57.256907 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.256457 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k292x" event={"ID":"63698879-bca0-4fb4-944b-3151be7fbc11","Type":"ContainerStarted","Data":"35dca7b0369cd134752812ab0c4939931a4379bbcf935a19adb24ccd1dc59ce1"} Jul 06 18:20:57.256907 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.256504 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k292x" event={"ID":"63698879-bca0-4fb4-944b-3151be7fbc11","Type":"ContainerStarted","Data":"b09b128814498a5e910a48843935156fa93365576c4620480991f3decdec0e12"} Jul 06 18:20:57.260487 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.260449 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:57.260828 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.260809 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qj66f" event={"ID":"9bda0943-9d88-4367-be22-9ec970388f56","Type":"ContainerStarted","Data":"c69b8a6ef9f985dfff01053ac0bd415267aedaa785ddecf9be7f90119e7d9552"} Jul 06 18:20:57.332037 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.331975 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-sxg68" podStartSLOduration=4.546035934 podStartE2EDuration="6.331940098s" totalImagesPullingTime="1.785904164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:20:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:57.307159132 +0000 UTC m=+74.033588101" watchObservedRunningTime="2026-07-06 18:20:57.331940098 +0000 UTC m=+74.058369057" Jul 06 18:20:57.356237 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.356171 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-k292x" podStartSLOduration=5.388964842 podStartE2EDuration="6.356154465s" totalImagesPullingTime="894.011695ms" totalInitContainerRuntime="73.177928ms" isStatefulPod=true podCreationTimestamp="2026-07-06 18:20:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:57.355406824 +0000 UTC m=+74.081835783" watchObservedRunningTime="2026-07-06 18:20:57.356154465 +0000 UTC m=+74.082583422" Jul 06 18:20:57.356734 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.356697 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qj66f" podStartSLOduration=4.724708951 podStartE2EDuration="6.35668872s" totalImagesPullingTime="1.631979769s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:20:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:57.330294027 +0000 UTC m=+74.056722997" watchObservedRunningTime="2026-07-06 18:20:57.35668872 +0000 UTC m=+74.083117677" Jul 06 18:20:57.387234 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.387202 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6699bc67c9-zwmdh"] Jul 06 18:20:57.416770 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.416734 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-754859f75b-2779n"] Jul 06 18:20:57.419564 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.419526 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-754859f75b-2779n" Jul 06 18:20:57.441624 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.441599 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-754859f75b-2779n"] Jul 06 18:20:57.481389 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.481342 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-service-ca\") pod \"console-754859f75b-2779n\" (UID: \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\") " pod="openshift-console/console-754859f75b-2779n" Jul 06 18:20:57.481389 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.481390 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ct7z\" (UniqueName: \"kubernetes.io/projected/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-kube-api-access-9ct7z\") pod \"console-754859f75b-2779n\" (UID: \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\") " pod="openshift-console/console-754859f75b-2779n" Jul 06 18:20:57.481591 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.481434 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-console-serving-cert\") pod \"console-754859f75b-2779n\" (UID: \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\") " pod="openshift-console/console-754859f75b-2779n" Jul 06 18:20:57.481591 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.481467 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-console-oauth-config\") pod \"console-754859f75b-2779n\" (UID: \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\") " pod="openshift-console/console-754859f75b-2779n" Jul 06 18:20:57.481591 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.481552 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-oauth-serving-cert\") pod \"console-754859f75b-2779n\" (UID: \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\") " pod="openshift-console/console-754859f75b-2779n" Jul 06 18:20:57.481711 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.481614 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-console-config\") pod \"console-754859f75b-2779n\" (UID: \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\") " pod="openshift-console/console-754859f75b-2779n" Jul 06 18:20:57.481711 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.481658 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-trusted-ca-bundle\") pod \"console-754859f75b-2779n\" (UID: \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\") " pod="openshift-console/console-754859f75b-2779n" Jul 06 18:20:57.583585 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.583478 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-console-config\") pod \"console-754859f75b-2779n\" (UID: \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\") " pod="openshift-console/console-754859f75b-2779n" Jul 06 18:20:57.583585 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.583549 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-trusted-ca-bundle\") pod \"console-754859f75b-2779n\" (UID: \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\") " pod="openshift-console/console-754859f75b-2779n" Jul 06 18:20:57.583802 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.583613 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-service-ca\") pod \"console-754859f75b-2779n\" (UID: \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\") " pod="openshift-console/console-754859f75b-2779n" Jul 06 18:20:57.583802 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.583654 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ct7z\" (UniqueName: \"kubernetes.io/projected/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-kube-api-access-9ct7z\") pod \"console-754859f75b-2779n\" (UID: \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\") " pod="openshift-console/console-754859f75b-2779n" Jul 06 18:20:57.583802 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.583687 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-console-serving-cert\") pod \"console-754859f75b-2779n\" (UID: \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\") " pod="openshift-console/console-754859f75b-2779n" Jul 06 18:20:57.583802 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.583731 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-console-oauth-config\") pod \"console-754859f75b-2779n\" (UID: \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\") " pod="openshift-console/console-754859f75b-2779n" Jul 06 18:20:57.583802 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.583767 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-oauth-serving-cert\") pod \"console-754859f75b-2779n\" (UID: \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\") " pod="openshift-console/console-754859f75b-2779n" Jul 06 18:20:57.584429 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.584386 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-console-config\") pod \"console-754859f75b-2779n\" (UID: \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\") " pod="openshift-console/console-754859f75b-2779n" Jul 06 18:20:57.584560 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.584540 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-service-ca\") pod \"console-754859f75b-2779n\" (UID: \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\") " pod="openshift-console/console-754859f75b-2779n" Jul 06 18:20:57.584621 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.584576 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-trusted-ca-bundle\") pod \"console-754859f75b-2779n\" (UID: \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\") " pod="openshift-console/console-754859f75b-2779n" Jul 06 18:20:57.585091 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.585068 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-oauth-serving-cert\") pod \"console-754859f75b-2779n\" (UID: \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\") " pod="openshift-console/console-754859f75b-2779n" Jul 06 18:20:57.586347 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.586328 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-console-serving-cert\") pod \"console-754859f75b-2779n\" (UID: \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\") " pod="openshift-console/console-754859f75b-2779n" Jul 06 18:20:57.586396 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.586381 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-console-oauth-config\") pod \"console-754859f75b-2779n\" (UID: \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\") " pod="openshift-console/console-754859f75b-2779n" Jul 06 18:20:57.593755 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.593720 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ct7z\" (UniqueName: \"kubernetes.io/projected/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-kube-api-access-9ct7z\") pod \"console-754859f75b-2779n\" (UID: \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\") " pod="openshift-console/console-754859f75b-2779n" Jul 06 18:20:57.730397 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.730358 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-754859f75b-2779n" Jul 06 18:20:57.901843 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:57.901806 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-754859f75b-2779n"] Jul 06 18:20:58.127194 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:58.126969 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6cfff669f4-6fmtq" Jul 06 18:20:58.272994 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:58.272125 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" event={"ID":"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6","Type":"ContainerStarted","Data":"2b8c809f28f32a4aa6cd8f45cfb81f321ab9bce1dd03b82bc8fff3833c8a80bd"} Jul 06 18:20:58.290689 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:58.290459 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-664b87676-6zq6q" Jul 06 18:20:58.290689 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:58.290525 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-664b87676-6zq6q" Jul 06 18:20:58.297901 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:58.297627 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-664b87676-6zq6q" Jul 06 18:20:58.329207 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:58.329120 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:58.400255 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:58.400226 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:58.886540 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:58.886468 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:58.906114 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:20:58.906087 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a635941_040e_4d5d_b8d8_6c04c6a4d2f0.slice/crio-71be6f63c63dd332e664d9a3f66ba095a0abf2dad9fa1e7b36a159fcad110aac WatchSource:0}: Error finding container 71be6f63c63dd332e664d9a3f66ba095a0abf2dad9fa1e7b36a159fcad110aac: Status 404 returned error can't find the container with id 71be6f63c63dd332e664d9a3f66ba095a0abf2dad9fa1e7b36a159fcad110aac Jul 06 18:20:58.987829 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:58.987787 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:59.278672 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:59.278622 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-754859f75b-2779n" event={"ID":"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0","Type":"ContainerStarted","Data":"71be6f63c63dd332e664d9a3f66ba095a0abf2dad9fa1e7b36a159fcad110aac"} Jul 06 18:20:59.282715 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:59.282691 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-664b87676-6zq6q" Jul 06 18:20:59.413829 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:59.413791 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:20:59.515449 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:20:59.515362 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:21:00.294111 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:00.294074 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6754684457-85h2d" event={"ID":"db2ee9dc-cdec-4fd5-abc4-d049d53b829a","Type":"ContainerStarted","Data":"14a51fb2ba5941476da20c476501fcdf3841eb344b16baed07128c2d26e45483"} Jul 06 18:21:00.298739 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:00.298562 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dde180b5-8bfa-42de-b9d0-aa713d3c7855","Type":"ContainerStarted","Data":"d238f57264a7dac0d6d9c00b6218327985fea3b4be0b70b5825881f46e792e4a"} Jul 06 18:21:00.300834 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:00.300803 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-v67rj" event={"ID":"57e592f6-5606-4945-b8e4-87fa68fa0977","Type":"ContainerStarted","Data":"45c8316a0b19f56dbb83b5175d3cdb7d44e0cc7389979a2cdb0151f4ddbd322e"} Jul 06 18:21:00.302455 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:00.302430 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-v67rj" Jul 06 18:21:00.305228 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:00.305155 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" event={"ID":"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6","Type":"ContainerStarted","Data":"66e7b0fdf64c0fed430495a297a8f96cc6d8b8c244fd895505238961522201eb"} Jul 06 18:21:00.307463 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:00.307407 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-754859f75b-2779n" event={"ID":"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0","Type":"ContainerStarted","Data":"a74bcd52390d1a34e2ee440bb96d7c88553bb453fa2a865675f06628f6e17a5b"} Jul 06 18:21:00.309054 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:00.309038 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-v67rj" Jul 06 18:21:00.319112 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:00.319062 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6754684457-85h2d" podStartSLOduration=2.249340451 podStartE2EDuration="5.31904215s" totalImagesPullingTime="3.069701699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:20:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:21:00.317799981 +0000 UTC m=+77.044228951" watchObservedRunningTime="2026-07-06 18:21:00.31904215 +0000 UTC m=+77.045471109" Jul 06 18:21:00.340661 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:00.340609 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-v67rj" podStartSLOduration=1.689355513 podStartE2EDuration="5.340588713s" totalImagesPullingTime="3.6512332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:20:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:21:00.339477395 +0000 UTC m=+77.065906354" watchObservedRunningTime="2026-07-06 18:21:00.340588713 +0000 UTC m=+77.067017731" Jul 06 18:21:00.363371 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:00.363226 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-754859f75b-2779n" podStartSLOduration=3.36320253 podStartE2EDuration="3.36320253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:20:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:21:00.359581245 +0000 UTC m=+77.086010207" watchObservedRunningTime="2026-07-06 18:21:00.36320253 +0000 UTC m=+77.089631489" Jul 06 18:21:01.316105 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:01.316065 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dde180b5-8bfa-42de-b9d0-aa713d3c7855","Type":"ContainerStarted","Data":"9f74323baa00bbab4f926b1665ff60f6097fd3c59f28c8bb4fc24e4654b393ab"} Jul 06 18:21:01.316564 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:01.316139 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dde180b5-8bfa-42de-b9d0-aa713d3c7855","Type":"ContainerStarted","Data":"5b214e1e238650ab22ec30d7e1ade48edfbb59a3b9db73b895e353a4e0b39104"} Jul 06 18:21:01.316564 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:01.316179 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dde180b5-8bfa-42de-b9d0-aa713d3c7855","Type":"ContainerStarted","Data":"4dfdcd94e24cdbd95bb0bd5ebec91cd073969288e443698402a1448dae103441"} Jul 06 18:21:01.316564 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:01.316208 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dde180b5-8bfa-42de-b9d0-aa713d3c7855","Type":"ContainerStarted","Data":"634cb858fe4956bf4b00c574dab7b7d20a820c03b8e280c7b8f08b35036404a9"} Jul 06 18:21:01.316564 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:01.316248 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dde180b5-8bfa-42de-b9d0-aa713d3c7855","Type":"ContainerStarted","Data":"703c44e3cb2fa43435d73c0299b4574b8ee19ddbcd8f4ee5cbaab3cb413bbe51"} Jul 06 18:21:01.319106 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:01.319079 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" event={"ID":"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6","Type":"ContainerStarted","Data":"8275c657669def9ec714185c528247d8157bb8f435517dda422f9b96ffa02f8f"} Jul 06 18:21:01.319232 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:01.319143 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" event={"ID":"f34f8e8d-3be3-4770-b3d8-9db846d5dcc6","Type":"ContainerStarted","Data":"dbb296141291deaf32b878bd7e06166a01db648d9e8bd2869093cf11054b69e2"} Jul 06 18:21:01.319300 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:01.319281 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" Jul 06 18:21:01.345695 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:01.345643 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.438287281 podStartE2EDuration="9.345625223s" totalImagesPullingTime="4.879471202s" totalInitContainerRuntime="27.86674ms" isStatefulPod=false podCreationTimestamp="2026-07-06 18:20:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:21:01.342780601 +0000 UTC m=+78.069209559" watchObservedRunningTime="2026-07-06 18:21:01.345625223 +0000 UTC m=+78.072054181" Jul 06 18:21:01.367598 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:01.367533 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" podStartSLOduration=2.931106037 podStartE2EDuration="8.367509412s" totalImagesPullingTime="5.436403375s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:20:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:21:01.364777599 +0000 UTC m=+78.091206584" watchObservedRunningTime="2026-07-06 18:21:01.367509412 +0000 UTC m=+78.093938372" Jul 06 18:21:01.593569 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:01.593479 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:21:01.649718 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:01.649677 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:21:02.329935 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:02.329906 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-85bfd88756-s6985" Jul 06 18:21:02.516673 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:02.516636 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:21:02.573564 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:02.573524 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:21:03.795861 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:03.795716 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-754859f75b-2779n"] Jul 06 18:21:03.868188 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:03.868155 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-586c9fc948-wjphf"] Jul 06 18:21:03.871027 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:03.870990 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-586c9fc948-wjphf" Jul 06 18:21:03.882301 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:03.882276 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-586c9fc948-wjphf"] Jul 06 18:21:03.940251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:03.940213 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b00fa2f2-9886-4c5a-9970-04465cea42dd-oauth-serving-cert\") pod \"console-586c9fc948-wjphf\" (UID: \"b00fa2f2-9886-4c5a-9970-04465cea42dd\") " pod="openshift-console/console-586c9fc948-wjphf" Jul 06 18:21:03.940251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:03.940249 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b00fa2f2-9886-4c5a-9970-04465cea42dd-console-oauth-config\") pod \"console-586c9fc948-wjphf\" (UID: \"b00fa2f2-9886-4c5a-9970-04465cea42dd\") " pod="openshift-console/console-586c9fc948-wjphf" Jul 06 18:21:03.940464 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:03.940270 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b00fa2f2-9886-4c5a-9970-04465cea42dd-console-serving-cert\") pod \"console-586c9fc948-wjphf\" (UID: \"b00fa2f2-9886-4c5a-9970-04465cea42dd\") " pod="openshift-console/console-586c9fc948-wjphf" Jul 06 18:21:03.940464 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:03.940339 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b00fa2f2-9886-4c5a-9970-04465cea42dd-console-config\") pod \"console-586c9fc948-wjphf\" (UID: \"b00fa2f2-9886-4c5a-9970-04465cea42dd\") " pod="openshift-console/console-586c9fc948-wjphf" Jul 06 18:21:03.940464 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:03.940367 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b00fa2f2-9886-4c5a-9970-04465cea42dd-trusted-ca-bundle\") pod \"console-586c9fc948-wjphf\" (UID: \"b00fa2f2-9886-4c5a-9970-04465cea42dd\") " pod="openshift-console/console-586c9fc948-wjphf" Jul 06 18:21:03.940464 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:03.940414 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pc42\" (UniqueName: \"kubernetes.io/projected/b00fa2f2-9886-4c5a-9970-04465cea42dd-kube-api-access-8pc42\") pod \"console-586c9fc948-wjphf\" (UID: \"b00fa2f2-9886-4c5a-9970-04465cea42dd\") " pod="openshift-console/console-586c9fc948-wjphf" Jul 06 18:21:03.940464 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:03.940458 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b00fa2f2-9886-4c5a-9970-04465cea42dd-service-ca\") pod \"console-586c9fc948-wjphf\" (UID: \"b00fa2f2-9886-4c5a-9970-04465cea42dd\") " pod="openshift-console/console-586c9fc948-wjphf" Jul 06 18:21:04.041610 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:04.041568 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b00fa2f2-9886-4c5a-9970-04465cea42dd-oauth-serving-cert\") pod \"console-586c9fc948-wjphf\" (UID: \"b00fa2f2-9886-4c5a-9970-04465cea42dd\") " pod="openshift-console/console-586c9fc948-wjphf" Jul 06 18:21:04.041610 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:04.041609 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b00fa2f2-9886-4c5a-9970-04465cea42dd-console-oauth-config\") pod \"console-586c9fc948-wjphf\" (UID: \"b00fa2f2-9886-4c5a-9970-04465cea42dd\") " pod="openshift-console/console-586c9fc948-wjphf" Jul 06 18:21:04.041814 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:04.041629 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b00fa2f2-9886-4c5a-9970-04465cea42dd-console-serving-cert\") pod \"console-586c9fc948-wjphf\" (UID: \"b00fa2f2-9886-4c5a-9970-04465cea42dd\") " pod="openshift-console/console-586c9fc948-wjphf" Jul 06 18:21:04.041814 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:04.041789 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b00fa2f2-9886-4c5a-9970-04465cea42dd-console-config\") pod \"console-586c9fc948-wjphf\" (UID: \"b00fa2f2-9886-4c5a-9970-04465cea42dd\") " pod="openshift-console/console-586c9fc948-wjphf" Jul 06 18:21:04.041814 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:04.041808 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b00fa2f2-9886-4c5a-9970-04465cea42dd-trusted-ca-bundle\") pod \"console-586c9fc948-wjphf\" (UID: \"b00fa2f2-9886-4c5a-9970-04465cea42dd\") " pod="openshift-console/console-586c9fc948-wjphf" Jul 06 18:21:04.041974 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:04.041839 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pc42\" (UniqueName: \"kubernetes.io/projected/b00fa2f2-9886-4c5a-9970-04465cea42dd-kube-api-access-8pc42\") pod \"console-586c9fc948-wjphf\" (UID: \"b00fa2f2-9886-4c5a-9970-04465cea42dd\") " pod="openshift-console/console-586c9fc948-wjphf" Jul 06 18:21:04.041974 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:04.041863 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b00fa2f2-9886-4c5a-9970-04465cea42dd-service-ca\") pod \"console-586c9fc948-wjphf\" (UID: \"b00fa2f2-9886-4c5a-9970-04465cea42dd\") " pod="openshift-console/console-586c9fc948-wjphf" Jul 06 18:21:04.042446 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:04.042413 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b00fa2f2-9886-4c5a-9970-04465cea42dd-oauth-serving-cert\") pod \"console-586c9fc948-wjphf\" (UID: \"b00fa2f2-9886-4c5a-9970-04465cea42dd\") " pod="openshift-console/console-586c9fc948-wjphf" Jul 06 18:21:04.042716 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:04.042485 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b00fa2f2-9886-4c5a-9970-04465cea42dd-console-config\") pod \"console-586c9fc948-wjphf\" (UID: \"b00fa2f2-9886-4c5a-9970-04465cea42dd\") " pod="openshift-console/console-586c9fc948-wjphf" Jul 06 18:21:04.042716 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:04.042532 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b00fa2f2-9886-4c5a-9970-04465cea42dd-service-ca\") pod \"console-586c9fc948-wjphf\" (UID: \"b00fa2f2-9886-4c5a-9970-04465cea42dd\") " pod="openshift-console/console-586c9fc948-wjphf" Jul 06 18:21:04.042981 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:04.042958 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b00fa2f2-9886-4c5a-9970-04465cea42dd-trusted-ca-bundle\") pod \"console-586c9fc948-wjphf\" (UID: \"b00fa2f2-9886-4c5a-9970-04465cea42dd\") " pod="openshift-console/console-586c9fc948-wjphf" Jul 06 18:21:04.044356 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:04.044338 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b00fa2f2-9886-4c5a-9970-04465cea42dd-console-serving-cert\") pod \"console-586c9fc948-wjphf\" (UID: \"b00fa2f2-9886-4c5a-9970-04465cea42dd\") " pod="openshift-console/console-586c9fc948-wjphf" Jul 06 18:21:04.044436 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:04.044412 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b00fa2f2-9886-4c5a-9970-04465cea42dd-console-oauth-config\") pod \"console-586c9fc948-wjphf\" (UID: \"b00fa2f2-9886-4c5a-9970-04465cea42dd\") " pod="openshift-console/console-586c9fc948-wjphf" Jul 06 18:21:04.051401 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:04.051336 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pc42\" (UniqueName: \"kubernetes.io/projected/b00fa2f2-9886-4c5a-9970-04465cea42dd-kube-api-access-8pc42\") pod \"console-586c9fc948-wjphf\" (UID: \"b00fa2f2-9886-4c5a-9970-04465cea42dd\") " pod="openshift-console/console-586c9fc948-wjphf" Jul 06 18:21:04.180746 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:04.180706 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-586c9fc948-wjphf" Jul 06 18:21:04.339736 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:04.339713 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-586c9fc948-wjphf"] Jul 06 18:21:04.342583 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:21:04.342557 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb00fa2f2_9886_4c5a_9970_04465cea42dd.slice/crio-5d38c652f4fe3bb7196ee32b8375e6b4283075db2725e59b4d4b6e36fda63ab0 WatchSource:0}: Error finding container 5d38c652f4fe3bb7196ee32b8375e6b4283075db2725e59b4d4b6e36fda63ab0: Status 404 returned error can't find the container with id 5d38c652f4fe3bb7196ee32b8375e6b4283075db2725e59b4d4b6e36fda63ab0 Jul 06 18:21:05.202517 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:05.202485 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:21:05.302150 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:05.302110 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:21:05.341740 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:05.341702 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-586c9fc948-wjphf" event={"ID":"b00fa2f2-9886-4c5a-9970-04465cea42dd","Type":"ContainerStarted","Data":"51d9a6f5c0cbc81cb715bafb8d1631648b8d1a1997ed221617f24691acbe0d32"} Jul 06 18:21:05.341969 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:05.341764 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-586c9fc948-wjphf" event={"ID":"b00fa2f2-9886-4c5a-9970-04465cea42dd","Type":"ContainerStarted","Data":"5d38c652f4fe3bb7196ee32b8375e6b4283075db2725e59b4d4b6e36fda63ab0"} Jul 06 18:21:05.361143 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:05.361089 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-586c9fc948-wjphf" podStartSLOduration=2.361073286 podStartE2EDuration="2.361073286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:21:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:21:05.359325741 +0000 UTC m=+82.085754698" watchObservedRunningTime="2026-07-06 18:21:05.361073286 +0000 UTC m=+82.087502243" Jul 06 18:21:07.731408 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:07.731376 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-754859f75b-2779n" Jul 06 18:21:07.926447 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:07.926408 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:21:08.026861 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:08.026793 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:21:10.134858 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:10.134820 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:21:10.232597 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:10.232566 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:21:11.573810 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:11.573774 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:21:11.633478 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:11.633448 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:21:14.181816 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:14.181775 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-586c9fc948-wjphf" Jul 06 18:21:14.182306 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:14.181844 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-586c9fc948-wjphf" Jul 06 18:21:14.187494 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:14.187472 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-586c9fc948-wjphf" Jul 06 18:21:14.383669 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:14.383640 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-586c9fc948-wjphf" Jul 06 18:21:14.432301 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:14.432223 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-664b87676-6zq6q"] Jul 06 18:21:14.527553 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:14.527512 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:21:14.585118 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:14.585089 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:21:14.639238 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:14.639203 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:21:15.745521 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:15.745484 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6754684457-85h2d" Jul 06 18:21:15.745931 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:15.745543 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6754684457-85h2d" Jul 06 18:21:22.413007 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:22.412963 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6699bc67c9-zwmdh" podUID="4a62d81e-42fa-4604-ab20-11130d5cd553" containerName="console" containerID="cri-o://b05f8f864308ab3bd8e408a072e52acb6d3a0df3f52e89b2aa93695f12efdcac" gracePeriod=15 Jul 06 18:21:23.183524 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:23.183497 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6699bc67c9-zwmdh" Jul 06 18:21:23.324307 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:23.324217 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a62d81e-42fa-4604-ab20-11130d5cd553-console-oauth-config\") pod \"4a62d81e-42fa-4604-ab20-11130d5cd553\" (UID: \"4a62d81e-42fa-4604-ab20-11130d5cd553\") " Jul 06 18:21:23.324307 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:23.324281 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6pgs\" (UniqueName: \"kubernetes.io/projected/4a62d81e-42fa-4604-ab20-11130d5cd553-kube-api-access-f6pgs\") pod \"4a62d81e-42fa-4604-ab20-11130d5cd553\" (UID: \"4a62d81e-42fa-4604-ab20-11130d5cd553\") " Jul 06 18:21:23.324537 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:23.324325 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a62d81e-42fa-4604-ab20-11130d5cd553-console-config\") pod \"4a62d81e-42fa-4604-ab20-11130d5cd553\" (UID: \"4a62d81e-42fa-4604-ab20-11130d5cd553\") " Jul 06 18:21:23.324537 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:23.324366 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a62d81e-42fa-4604-ab20-11130d5cd553-oauth-serving-cert\") pod \"4a62d81e-42fa-4604-ab20-11130d5cd553\" (UID: \"4a62d81e-42fa-4604-ab20-11130d5cd553\") " Jul 06 18:21:23.324537 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:23.324434 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a62d81e-42fa-4604-ab20-11130d5cd553-service-ca\") pod \"4a62d81e-42fa-4604-ab20-11130d5cd553\" (UID: \"4a62d81e-42fa-4604-ab20-11130d5cd553\") " Jul 06 18:21:23.324537 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:23.324460 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a62d81e-42fa-4604-ab20-11130d5cd553-console-serving-cert\") pod \"4a62d81e-42fa-4604-ab20-11130d5cd553\" (UID: \"4a62d81e-42fa-4604-ab20-11130d5cd553\") " Jul 06 18:21:23.324819 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:23.324792 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a62d81e-42fa-4604-ab20-11130d5cd553-console-config" (OuterVolumeSpecName: "console-config") pod "4a62d81e-42fa-4604-ab20-11130d5cd553" (UID: "4a62d81e-42fa-4604-ab20-11130d5cd553"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:21:23.324935 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:23.324903 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a62d81e-42fa-4604-ab20-11130d5cd553-service-ca" (OuterVolumeSpecName: "service-ca") pod "4a62d81e-42fa-4604-ab20-11130d5cd553" (UID: "4a62d81e-42fa-4604-ab20-11130d5cd553"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:21:23.325030 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:23.325006 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a62d81e-42fa-4604-ab20-11130d5cd553-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4a62d81e-42fa-4604-ab20-11130d5cd553" (UID: "4a62d81e-42fa-4604-ab20-11130d5cd553"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:21:23.326896 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:23.326859 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a62d81e-42fa-4604-ab20-11130d5cd553-kube-api-access-f6pgs" (OuterVolumeSpecName: "kube-api-access-f6pgs") pod "4a62d81e-42fa-4604-ab20-11130d5cd553" (UID: "4a62d81e-42fa-4604-ab20-11130d5cd553"). InnerVolumeSpecName "kube-api-access-f6pgs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:21:23.327185 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:23.327164 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a62d81e-42fa-4604-ab20-11130d5cd553-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4a62d81e-42fa-4604-ab20-11130d5cd553" (UID: "4a62d81e-42fa-4604-ab20-11130d5cd553"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:21:23.327270 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:23.327242 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a62d81e-42fa-4604-ab20-11130d5cd553-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4a62d81e-42fa-4604-ab20-11130d5cd553" (UID: "4a62d81e-42fa-4604-ab20-11130d5cd553"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:21:23.418017 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:23.417977 2566 generic.go:356] "Generic (PLEG): container finished" podID="4a62d81e-42fa-4604-ab20-11130d5cd553" containerID="b05f8f864308ab3bd8e408a072e52acb6d3a0df3f52e89b2aa93695f12efdcac" exitCode=0 Jul 06 18:21:23.418488 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:23.418022 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6699bc67c9-zwmdh" Jul 06 18:21:23.418488 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:23.418057 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6699bc67c9-zwmdh" event={"ID":"4a62d81e-42fa-4604-ab20-11130d5cd553","Type":"ContainerDied","Data":"b05f8f864308ab3bd8e408a072e52acb6d3a0df3f52e89b2aa93695f12efdcac"} Jul 06 18:21:23.418488 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:23.418101 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6699bc67c9-zwmdh" event={"ID":"4a62d81e-42fa-4604-ab20-11130d5cd553","Type":"ContainerDied","Data":"e3cb790038d33e39115dd49832644b5dfddf6452808cab8066f5adbeccb1c3fe"} Jul 06 18:21:23.418488 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:23.418123 2566 scope.go:117] "RemoveContainer" containerID="b05f8f864308ab3bd8e408a072e52acb6d3a0df3f52e89b2aa93695f12efdcac" Jul 06 18:21:23.425319 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:23.425282 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6pgs\" (UniqueName: \"kubernetes.io/projected/4a62d81e-42fa-4604-ab20-11130d5cd553-kube-api-access-f6pgs\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:21:23.425319 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:23.425312 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a62d81e-42fa-4604-ab20-11130d5cd553-console-config\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:21:23.425502 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:23.425327 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a62d81e-42fa-4604-ab20-11130d5cd553-oauth-serving-cert\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:21:23.425502 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:23.425338 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a62d81e-42fa-4604-ab20-11130d5cd553-service-ca\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:21:23.425502 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:23.425350 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a62d81e-42fa-4604-ab20-11130d5cd553-console-serving-cert\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:21:23.425502 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:23.425361 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a62d81e-42fa-4604-ab20-11130d5cd553-console-oauth-config\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:21:23.430111 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:23.430092 2566 scope.go:117] "RemoveContainer" containerID="b05f8f864308ab3bd8e408a072e52acb6d3a0df3f52e89b2aa93695f12efdcac" Jul 06 18:21:23.430412 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:21:23.430391 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b05f8f864308ab3bd8e408a072e52acb6d3a0df3f52e89b2aa93695f12efdcac\": container with ID starting with b05f8f864308ab3bd8e408a072e52acb6d3a0df3f52e89b2aa93695f12efdcac not found: ID does not exist" containerID="b05f8f864308ab3bd8e408a072e52acb6d3a0df3f52e89b2aa93695f12efdcac" Jul 06 18:21:23.430474 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:23.430424 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b05f8f864308ab3bd8e408a072e52acb6d3a0df3f52e89b2aa93695f12efdcac"} err="failed to get container status \"b05f8f864308ab3bd8e408a072e52acb6d3a0df3f52e89b2aa93695f12efdcac\": rpc error: code = NotFound desc = could not find container \"b05f8f864308ab3bd8e408a072e52acb6d3a0df3f52e89b2aa93695f12efdcac\": container with ID starting with b05f8f864308ab3bd8e408a072e52acb6d3a0df3f52e89b2aa93695f12efdcac not found: ID does not exist" Jul 06 18:21:23.443078 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:23.443046 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6699bc67c9-zwmdh"] Jul 06 18:21:23.446975 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:23.446953 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6699bc67c9-zwmdh"] Jul 06 18:21:23.848562 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:23.848527 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a62d81e-42fa-4604-ab20-11130d5cd553" path="/var/lib/kubelet/pods/4a62d81e-42fa-4604-ab20-11130d5cd553/volumes" Jul 06 18:21:25.151426 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:25.151389 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:21:25.230278 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:25.230252 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-n4j4l" Jul 06 18:21:25.256010 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:25.255982 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:21:28.815797 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:28.815757 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-754859f75b-2779n" podUID="6a635941-040e-4d5d-b8d8-6c04c6a4d2f0" containerName="console" containerID="cri-o://a74bcd52390d1a34e2ee440bb96d7c88553bb453fa2a865675f06628f6e17a5b" gracePeriod=15 Jul 06 18:21:29.098130 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.098097 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-754859f75b-2779n" Jul 06 18:21:29.278577 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.278535 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-service-ca\") pod \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\" (UID: \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\") " Jul 06 18:21:29.278577 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.278578 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ct7z\" (UniqueName: \"kubernetes.io/projected/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-kube-api-access-9ct7z\") pod \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\" (UID: \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\") " Jul 06 18:21:29.278846 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.278605 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-console-oauth-config\") pod \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\" (UID: \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\") " Jul 06 18:21:29.278846 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.278621 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-oauth-serving-cert\") pod \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\" (UID: \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\") " Jul 06 18:21:29.278846 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.278679 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-trusted-ca-bundle\") pod \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\" (UID: \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\") " Jul 06 18:21:29.278846 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.278726 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-console-serving-cert\") pod \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\" (UID: \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\") " Jul 06 18:21:29.278846 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.278765 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-console-config\") pod \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\" (UID: \"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0\") " Jul 06 18:21:29.279142 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.279098 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-service-ca" (OuterVolumeSpecName: "service-ca") pod "6a635941-040e-4d5d-b8d8-6c04c6a4d2f0" (UID: "6a635941-040e-4d5d-b8d8-6c04c6a4d2f0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:21:29.279198 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.279155 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6a635941-040e-4d5d-b8d8-6c04c6a4d2f0" (UID: "6a635941-040e-4d5d-b8d8-6c04c6a4d2f0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:21:29.279631 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.279362 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-console-config" (OuterVolumeSpecName: "console-config") pod "6a635941-040e-4d5d-b8d8-6c04c6a4d2f0" (UID: "6a635941-040e-4d5d-b8d8-6c04c6a4d2f0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:21:29.279631 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.279390 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6a635941-040e-4d5d-b8d8-6c04c6a4d2f0" (UID: "6a635941-040e-4d5d-b8d8-6c04c6a4d2f0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:21:29.281112 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.281087 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6a635941-040e-4d5d-b8d8-6c04c6a4d2f0" (UID: "6a635941-040e-4d5d-b8d8-6c04c6a4d2f0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:21:29.281200 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.281103 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6a635941-040e-4d5d-b8d8-6c04c6a4d2f0" (UID: "6a635941-040e-4d5d-b8d8-6c04c6a4d2f0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:21:29.281200 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.281172 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-kube-api-access-9ct7z" (OuterVolumeSpecName: "kube-api-access-9ct7z") pod "6a635941-040e-4d5d-b8d8-6c04c6a4d2f0" (UID: "6a635941-040e-4d5d-b8d8-6c04c6a4d2f0"). InnerVolumeSpecName "kube-api-access-9ct7z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:21:29.380168 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.380062 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-console-serving-cert\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:21:29.380168 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.380103 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-console-config\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:21:29.380168 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.380112 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-service-ca\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:21:29.380168 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.380121 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ct7z\" (UniqueName: \"kubernetes.io/projected/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-kube-api-access-9ct7z\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:21:29.380168 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.380130 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-console-oauth-config\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:21:29.380168 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.380137 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-oauth-serving-cert\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:21:29.380168 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.380146 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0-trusted-ca-bundle\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:21:29.443741 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.443711 2566 generic.go:356] "Generic (PLEG): container finished" podID="6a635941-040e-4d5d-b8d8-6c04c6a4d2f0" containerID="a74bcd52390d1a34e2ee440bb96d7c88553bb453fa2a865675f06628f6e17a5b" exitCode=0 Jul 06 18:21:29.443927 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.443748 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-754859f75b-2779n" event={"ID":"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0","Type":"ContainerDied","Data":"a74bcd52390d1a34e2ee440bb96d7c88553bb453fa2a865675f06628f6e17a5b"} Jul 06 18:21:29.443927 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.443769 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-754859f75b-2779n" Jul 06 18:21:29.443927 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.443814 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-754859f75b-2779n" event={"ID":"6a635941-040e-4d5d-b8d8-6c04c6a4d2f0","Type":"ContainerDied","Data":"71be6f63c63dd332e664d9a3f66ba095a0abf2dad9fa1e7b36a159fcad110aac"} Jul 06 18:21:29.443927 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.443834 2566 scope.go:117] "RemoveContainer" containerID="a74bcd52390d1a34e2ee440bb96d7c88553bb453fa2a865675f06628f6e17a5b" Jul 06 18:21:29.452695 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.452675 2566 scope.go:117] "RemoveContainer" containerID="a74bcd52390d1a34e2ee440bb96d7c88553bb453fa2a865675f06628f6e17a5b" Jul 06 18:21:29.453002 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:21:29.452982 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a74bcd52390d1a34e2ee440bb96d7c88553bb453fa2a865675f06628f6e17a5b\": container with ID starting with a74bcd52390d1a34e2ee440bb96d7c88553bb453fa2a865675f06628f6e17a5b not found: ID does not exist" containerID="a74bcd52390d1a34e2ee440bb96d7c88553bb453fa2a865675f06628f6e17a5b" Jul 06 18:21:29.453055 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.453012 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a74bcd52390d1a34e2ee440bb96d7c88553bb453fa2a865675f06628f6e17a5b"} err="failed to get container status \"a74bcd52390d1a34e2ee440bb96d7c88553bb453fa2a865675f06628f6e17a5b\": rpc error: code = NotFound desc = could not find container \"a74bcd52390d1a34e2ee440bb96d7c88553bb453fa2a865675f06628f6e17a5b\": container with ID starting with a74bcd52390d1a34e2ee440bb96d7c88553bb453fa2a865675f06628f6e17a5b not found: ID does not exist" Jul 06 18:21:29.466633 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.466598 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-754859f75b-2779n"] Jul 06 18:21:29.469230 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.469207 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-754859f75b-2779n"] Jul 06 18:21:29.849038 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:29.849004 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a635941-040e-4d5d-b8d8-6c04c6a4d2f0" path="/var/lib/kubelet/pods/6a635941-040e-4d5d-b8d8-6c04c6a4d2f0/volumes" Jul 06 18:21:32.349615 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:32.349587 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:21:32.450379 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:32.450348 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:21:32.503409 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:32.503366 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:21:34.470475 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:34.470445 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:21:35.751041 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:35.751010 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6754684457-85h2d" Jul 06 18:21:35.754837 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:35.754814 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6754684457-85h2d" Jul 06 18:21:39.453448 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:39.453383 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-664b87676-6zq6q" podUID="be5705f0-bde0-4b0f-a699-5f2b6b3d8018" containerName="console" containerID="cri-o://dd230413a7775ca5e2b8e38f4b7177015f2e29aa740c184278fc4be429b6098d" gracePeriod=15 Jul 06 18:21:39.696336 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:39.696305 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-664b87676-6zq6q" Jul 06 18:21:39.768745 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:39.768654 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6g82\" (UniqueName: \"kubernetes.io/projected/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-kube-api-access-n6g82\") pod \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\" (UID: \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\") " Jul 06 18:21:39.768745 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:39.768719 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-console-oauth-config\") pod \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\" (UID: \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\") " Jul 06 18:21:39.768745 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:39.768742 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-console-config\") pod \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\" (UID: \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\") " Jul 06 18:21:39.769049 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:39.768766 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-trusted-ca-bundle\") pod \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\" (UID: \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\") " Jul 06 18:21:39.769049 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:39.768800 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-console-serving-cert\") pod \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\" (UID: \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\") " Jul 06 18:21:39.769049 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:39.768826 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-oauth-serving-cert\") pod \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\" (UID: \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\") " Jul 06 18:21:39.769198 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:39.769152 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-service-ca\") pod \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\" (UID: \"be5705f0-bde0-4b0f-a699-5f2b6b3d8018\") " Jul 06 18:21:39.769317 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:39.769253 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "be5705f0-bde0-4b0f-a699-5f2b6b3d8018" (UID: "be5705f0-bde0-4b0f-a699-5f2b6b3d8018"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:21:39.769317 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:39.769273 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-console-config" (OuterVolumeSpecName: "console-config") pod "be5705f0-bde0-4b0f-a699-5f2b6b3d8018" (UID: "be5705f0-bde0-4b0f-a699-5f2b6b3d8018"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:21:39.769505 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:39.769341 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-service-ca" (OuterVolumeSpecName: "service-ca") pod "be5705f0-bde0-4b0f-a699-5f2b6b3d8018" (UID: "be5705f0-bde0-4b0f-a699-5f2b6b3d8018"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:21:39.769505 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:39.769450 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "be5705f0-bde0-4b0f-a699-5f2b6b3d8018" (UID: "be5705f0-bde0-4b0f-a699-5f2b6b3d8018"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:21:39.769688 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:39.769669 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-oauth-serving-cert\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:21:39.769724 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:39.769696 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-service-ca\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:21:39.769724 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:39.769713 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-console-config\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:21:39.769785 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:39.769727 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-trusted-ca-bundle\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:21:39.771192 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:39.771171 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-kube-api-access-n6g82" (OuterVolumeSpecName: "kube-api-access-n6g82") pod "be5705f0-bde0-4b0f-a699-5f2b6b3d8018" (UID: "be5705f0-bde0-4b0f-a699-5f2b6b3d8018"). InnerVolumeSpecName "kube-api-access-n6g82". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:21:39.771290 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:39.771206 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "be5705f0-bde0-4b0f-a699-5f2b6b3d8018" (UID: "be5705f0-bde0-4b0f-a699-5f2b6b3d8018"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:21:39.771290 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:39.771271 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "be5705f0-bde0-4b0f-a699-5f2b6b3d8018" (UID: "be5705f0-bde0-4b0f-a699-5f2b6b3d8018"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:21:39.870603 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:39.870579 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6g82\" (UniqueName: \"kubernetes.io/projected/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-kube-api-access-n6g82\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:21:39.870603 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:39.870603 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-console-oauth-config\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:21:39.870741 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:39.870613 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/be5705f0-bde0-4b0f-a699-5f2b6b3d8018-console-serving-cert\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:21:40.494487 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:40.494449 2566 generic.go:356] "Generic (PLEG): container finished" podID="be5705f0-bde0-4b0f-a699-5f2b6b3d8018" containerID="dd230413a7775ca5e2b8e38f4b7177015f2e29aa740c184278fc4be429b6098d" exitCode=0 Jul 06 18:21:40.494941 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:40.494493 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-664b87676-6zq6q" Jul 06 18:21:40.494941 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:40.494500 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-664b87676-6zq6q" event={"ID":"be5705f0-bde0-4b0f-a699-5f2b6b3d8018","Type":"ContainerDied","Data":"dd230413a7775ca5e2b8e38f4b7177015f2e29aa740c184278fc4be429b6098d"} Jul 06 18:21:40.494941 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:40.494550 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-664b87676-6zq6q" event={"ID":"be5705f0-bde0-4b0f-a699-5f2b6b3d8018","Type":"ContainerDied","Data":"ecfe1c0e132e3f7d8e1972d556523814e29be64b74533ffa1a03c74242bb8bf4"} Jul 06 18:21:40.494941 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:40.494571 2566 scope.go:117] "RemoveContainer" containerID="dd230413a7775ca5e2b8e38f4b7177015f2e29aa740c184278fc4be429b6098d" Jul 06 18:21:40.503055 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:40.503035 2566 scope.go:117] "RemoveContainer" containerID="dd230413a7775ca5e2b8e38f4b7177015f2e29aa740c184278fc4be429b6098d" Jul 06 18:21:40.503316 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:21:40.503295 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd230413a7775ca5e2b8e38f4b7177015f2e29aa740c184278fc4be429b6098d\": container with ID starting with dd230413a7775ca5e2b8e38f4b7177015f2e29aa740c184278fc4be429b6098d not found: ID does not exist" containerID="dd230413a7775ca5e2b8e38f4b7177015f2e29aa740c184278fc4be429b6098d" Jul 06 18:21:40.503374 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:40.503329 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd230413a7775ca5e2b8e38f4b7177015f2e29aa740c184278fc4be429b6098d"} err="failed to get container status \"dd230413a7775ca5e2b8e38f4b7177015f2e29aa740c184278fc4be429b6098d\": rpc error: code = NotFound desc = could not find container \"dd230413a7775ca5e2b8e38f4b7177015f2e29aa740c184278fc4be429b6098d\": container with ID starting with dd230413a7775ca5e2b8e38f4b7177015f2e29aa740c184278fc4be429b6098d not found: ID does not exist" Jul 06 18:21:40.511222 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:40.511196 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-664b87676-6zq6q"] Jul 06 18:21:40.515123 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:40.515102 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-664b87676-6zq6q"] Jul 06 18:21:41.851314 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:21:41.851279 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be5705f0-bde0-4b0f-a699-5f2b6b3d8018" path="/var/lib/kubelet/pods/be5705f0-bde0-4b0f-a699-5f2b6b3d8018/volumes" Jul 06 18:22:07.165268 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:07.165229 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:22:07.264213 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:07.264177 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:22:11.417125 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:11.417088 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 18:22:11.417573 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:11.417395 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerName="alertmanager" containerID="cri-o://d238f57264a7dac0d6d9c00b6218327985fea3b4be0b70b5825881f46e792e4a" gracePeriod=120 Jul 06 18:22:11.417573 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:11.417490 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerName="kube-rbac-proxy-web" containerID="cri-o://634cb858fe4956bf4b00c574dab7b7d20a820c03b8e280c7b8f08b35036404a9" gracePeriod=120 Jul 06 18:22:11.417573 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:11.417475 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerName="kube-rbac-proxy-metric" containerID="cri-o://5b214e1e238650ab22ec30d7e1ade48edfbb59a3b9db73b895e353a4e0b39104" gracePeriod=120 Jul 06 18:22:11.417573 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:11.417522 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerName="prom-label-proxy" containerID="cri-o://9f74323baa00bbab4f926b1665ff60f6097fd3c59f28c8bb4fc24e4654b393ab" gracePeriod=120 Jul 06 18:22:11.417573 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:11.417510 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerName="config-reloader" containerID="cri-o://703c44e3cb2fa43435d73c0299b4574b8ee19ddbcd8f4ee5cbaab3cb413bbe51" gracePeriod=120 Jul 06 18:22:11.417828 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:11.417497 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerName="kube-rbac-proxy" containerID="cri-o://4dfdcd94e24cdbd95bb0bd5ebec91cd073969288e443698402a1448dae103441" gracePeriod=120 Jul 06 18:22:11.618766 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:11.618731 2566 generic.go:356] "Generic (PLEG): container finished" podID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerID="9f74323baa00bbab4f926b1665ff60f6097fd3c59f28c8bb4fc24e4654b393ab" exitCode=0 Jul 06 18:22:11.618766 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:11.618756 2566 generic.go:356] "Generic (PLEG): container finished" podID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerID="4dfdcd94e24cdbd95bb0bd5ebec91cd073969288e443698402a1448dae103441" exitCode=0 Jul 06 18:22:11.618766 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:11.618763 2566 generic.go:356] "Generic (PLEG): container finished" podID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerID="703c44e3cb2fa43435d73c0299b4574b8ee19ddbcd8f4ee5cbaab3cb413bbe51" exitCode=0 Jul 06 18:22:11.618766 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:11.618770 2566 generic.go:356] "Generic (PLEG): container finished" podID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerID="d238f57264a7dac0d6d9c00b6218327985fea3b4be0b70b5825881f46e792e4a" exitCode=0 Jul 06 18:22:11.619049 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:11.618815 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dde180b5-8bfa-42de-b9d0-aa713d3c7855","Type":"ContainerDied","Data":"9f74323baa00bbab4f926b1665ff60f6097fd3c59f28c8bb4fc24e4654b393ab"} Jul 06 18:22:11.619049 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:11.618894 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dde180b5-8bfa-42de-b9d0-aa713d3c7855","Type":"ContainerDied","Data":"4dfdcd94e24cdbd95bb0bd5ebec91cd073969288e443698402a1448dae103441"} Jul 06 18:22:11.619049 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:11.618933 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dde180b5-8bfa-42de-b9d0-aa713d3c7855","Type":"ContainerDied","Data":"703c44e3cb2fa43435d73c0299b4574b8ee19ddbcd8f4ee5cbaab3cb413bbe51"} Jul 06 18:22:11.619049 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:11.618959 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dde180b5-8bfa-42de-b9d0-aa713d3c7855","Type":"ContainerDied","Data":"d238f57264a7dac0d6d9c00b6218327985fea3b4be0b70b5825881f46e792e4a"} Jul 06 18:22:12.631060 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.631028 2566 generic.go:356] "Generic (PLEG): container finished" podID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerID="5b214e1e238650ab22ec30d7e1ade48edfbb59a3b9db73b895e353a4e0b39104" exitCode=0 Jul 06 18:22:12.631060 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.631054 2566 generic.go:356] "Generic (PLEG): container finished" podID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerID="634cb858fe4956bf4b00c574dab7b7d20a820c03b8e280c7b8f08b35036404a9" exitCode=0 Jul 06 18:22:12.631500 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.631105 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dde180b5-8bfa-42de-b9d0-aa713d3c7855","Type":"ContainerDied","Data":"5b214e1e238650ab22ec30d7e1ade48edfbb59a3b9db73b895e353a4e0b39104"} Jul 06 18:22:12.631500 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.631222 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dde180b5-8bfa-42de-b9d0-aa713d3c7855","Type":"ContainerDied","Data":"634cb858fe4956bf4b00c574dab7b7d20a820c03b8e280c7b8f08b35036404a9"} Jul 06 18:22:12.662250 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.662229 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:12.761770 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.761680 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-web-config\") pod \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " Jul 06 18:22:12.761770 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.761734 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4vkk\" (UniqueName: \"kubernetes.io/projected/dde180b5-8bfa-42de-b9d0-aa713d3c7855-kube-api-access-k4vkk\") pod \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " Jul 06 18:22:12.761770 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.761766 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde180b5-8bfa-42de-b9d0-aa713d3c7855-alertmanager-trusted-ca-bundle\") pod \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " Jul 06 18:22:12.762074 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.761785 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dde180b5-8bfa-42de-b9d0-aa713d3c7855-tls-assets\") pod \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " Jul 06 18:22:12.762074 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.761802 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-secret-alertmanager-kube-rbac-proxy-web\") pod \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " Jul 06 18:22:12.762074 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.761830 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-config-volume\") pod \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " Jul 06 18:22:12.762280 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.762253 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dde180b5-8bfa-42de-b9d0-aa713d3c7855-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "dde180b5-8bfa-42de-b9d0-aa713d3c7855" (UID: "dde180b5-8bfa-42de-b9d0-aa713d3c7855"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:22:12.762353 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.762255 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dde180b5-8bfa-42de-b9d0-aa713d3c7855-metrics-client-ca\") pod \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " Jul 06 18:22:12.762407 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.762359 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dde180b5-8bfa-42de-b9d0-aa713d3c7855-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "dde180b5-8bfa-42de-b9d0-aa713d3c7855" (UID: "dde180b5-8bfa-42de-b9d0-aa713d3c7855"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:22:12.762518 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.762499 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-secret-alertmanager-kube-rbac-proxy\") pod \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " Jul 06 18:22:12.762590 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.762558 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dde180b5-8bfa-42de-b9d0-aa713d3c7855-alertmanager-main-db\") pod \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " Jul 06 18:22:12.762647 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.762591 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-secret-alertmanager-main-tls\") pod \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " Jul 06 18:22:12.762647 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.762625 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-cluster-tls-config\") pod \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " Jul 06 18:22:12.762755 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.762655 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dde180b5-8bfa-42de-b9d0-aa713d3c7855-config-out\") pod \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " Jul 06 18:22:12.762755 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.762723 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-secret-alertmanager-kube-rbac-proxy-metric\") pod \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\" (UID: \"dde180b5-8bfa-42de-b9d0-aa713d3c7855\") " Jul 06 18:22:12.763489 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.763259 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dde180b5-8bfa-42de-b9d0-aa713d3c7855-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "dde180b5-8bfa-42de-b9d0-aa713d3c7855" (UID: "dde180b5-8bfa-42de-b9d0-aa713d3c7855"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:22:12.763894 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.763804 2566 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dde180b5-8bfa-42de-b9d0-aa713d3c7855-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:22:12.763894 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.763832 2566 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dde180b5-8bfa-42de-b9d0-aa713d3c7855-metrics-client-ca\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:22:12.763894 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.763844 2566 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dde180b5-8bfa-42de-b9d0-aa713d3c7855-alertmanager-main-db\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:22:12.765784 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.765752 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "dde180b5-8bfa-42de-b9d0-aa713d3c7855" (UID: "dde180b5-8bfa-42de-b9d0-aa713d3c7855"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:22:12.765921 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.765895 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dde180b5-8bfa-42de-b9d0-aa713d3c7855-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "dde180b5-8bfa-42de-b9d0-aa713d3c7855" (UID: "dde180b5-8bfa-42de-b9d0-aa713d3c7855"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:22:12.765990 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.765895 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-config-volume" (OuterVolumeSpecName: "config-volume") pod "dde180b5-8bfa-42de-b9d0-aa713d3c7855" (UID: "dde180b5-8bfa-42de-b9d0-aa713d3c7855"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:22:12.766154 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.766113 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "dde180b5-8bfa-42de-b9d0-aa713d3c7855" (UID: "dde180b5-8bfa-42de-b9d0-aa713d3c7855"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:22:12.766227 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.766121 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dde180b5-8bfa-42de-b9d0-aa713d3c7855-kube-api-access-k4vkk" (OuterVolumeSpecName: "kube-api-access-k4vkk") pod "dde180b5-8bfa-42de-b9d0-aa713d3c7855" (UID: "dde180b5-8bfa-42de-b9d0-aa713d3c7855"). InnerVolumeSpecName "kube-api-access-k4vkk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:22:12.766484 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.766456 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dde180b5-8bfa-42de-b9d0-aa713d3c7855-config-out" (OuterVolumeSpecName: "config-out") pod "dde180b5-8bfa-42de-b9d0-aa713d3c7855" (UID: "dde180b5-8bfa-42de-b9d0-aa713d3c7855"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:22:12.766589 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.766542 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "dde180b5-8bfa-42de-b9d0-aa713d3c7855" (UID: "dde180b5-8bfa-42de-b9d0-aa713d3c7855"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:22:12.766935 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.766848 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "dde180b5-8bfa-42de-b9d0-aa713d3c7855" (UID: "dde180b5-8bfa-42de-b9d0-aa713d3c7855"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:22:12.770531 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.770509 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "dde180b5-8bfa-42de-b9d0-aa713d3c7855" (UID: "dde180b5-8bfa-42de-b9d0-aa713d3c7855"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:22:12.776903 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.776841 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-web-config" (OuterVolumeSpecName: "web-config") pod "dde180b5-8bfa-42de-b9d0-aa713d3c7855" (UID: "dde180b5-8bfa-42de-b9d0-aa713d3c7855"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:22:12.864378 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.864341 2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-secret-alertmanager-main-tls\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:22:12.864378 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.864368 2566 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-cluster-tls-config\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:22:12.864378 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.864378 2566 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dde180b5-8bfa-42de-b9d0-aa713d3c7855-config-out\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:22:12.864594 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.864389 2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:22:12.864594 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.864399 2566 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-web-config\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:22:12.864594 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.864406 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4vkk\" (UniqueName: \"kubernetes.io/projected/dde180b5-8bfa-42de-b9d0-aa713d3c7855-kube-api-access-k4vkk\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:22:12.864594 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.864414 2566 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dde180b5-8bfa-42de-b9d0-aa713d3c7855-tls-assets\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:22:12.864594 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.864423 2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:22:12.864594 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.864430 2566 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-config-volume\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:22:12.864594 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:12.864438 2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dde180b5-8bfa-42de-b9d0-aa713d3c7855-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:22:13.637947 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.637905 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dde180b5-8bfa-42de-b9d0-aa713d3c7855","Type":"ContainerDied","Data":"8f8a909508a07a56ccbb6ac022e31eafbe527678ef890bc059de273a7d107567"} Jul 06 18:22:13.638380 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.637987 2566 scope.go:117] "RemoveContainer" containerID="9f74323baa00bbab4f926b1665ff60f6097fd3c59f28c8bb4fc24e4654b393ab" Jul 06 18:22:13.638380 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.637921 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.648196 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.648172 2566 scope.go:117] "RemoveContainer" containerID="5b214e1e238650ab22ec30d7e1ade48edfbb59a3b9db73b895e353a4e0b39104" Jul 06 18:22:13.655705 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.655688 2566 scope.go:117] "RemoveContainer" containerID="4dfdcd94e24cdbd95bb0bd5ebec91cd073969288e443698402a1448dae103441" Jul 06 18:22:13.662446 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.662430 2566 scope.go:117] "RemoveContainer" containerID="634cb858fe4956bf4b00c574dab7b7d20a820c03b8e280c7b8f08b35036404a9" Jul 06 18:22:13.666464 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.666441 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 18:22:13.671514 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.671494 2566 scope.go:117] "RemoveContainer" containerID="703c44e3cb2fa43435d73c0299b4574b8ee19ddbcd8f4ee5cbaab3cb413bbe51" Jul 06 18:22:13.671771 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.671752 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 18:22:13.678766 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.678745 2566 scope.go:117] "RemoveContainer" containerID="d238f57264a7dac0d6d9c00b6218327985fea3b4be0b70b5825881f46e792e4a" Jul 06 18:22:13.685720 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.685704 2566 scope.go:117] "RemoveContainer" containerID="140fc9918bb72618333e263c87914003e984c718107f5bf626580029eaffff30" Jul 06 18:22:13.704321 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704298 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 18:22:13.704590 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704576 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a62d81e-42fa-4604-ab20-11130d5cd553" containerName="console" Jul 06 18:22:13.704590 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704589 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a62d81e-42fa-4604-ab20-11130d5cd553" containerName="console" Jul 06 18:22:13.704714 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704598 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerName="kube-rbac-proxy-metric" Jul 06 18:22:13.704714 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704604 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerName="kube-rbac-proxy-metric" Jul 06 18:22:13.704714 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704611 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerName="alertmanager" Jul 06 18:22:13.704714 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704616 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerName="alertmanager" Jul 06 18:22:13.704714 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704621 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerName="prom-label-proxy" Jul 06 18:22:13.704714 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704626 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerName="prom-label-proxy" Jul 06 18:22:13.704714 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704634 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerName="init-config-reloader" Jul 06 18:22:13.704714 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704640 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerName="init-config-reloader" Jul 06 18:22:13.704714 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704649 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be5705f0-bde0-4b0f-a699-5f2b6b3d8018" containerName="console" Jul 06 18:22:13.704714 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704654 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="be5705f0-bde0-4b0f-a699-5f2b6b3d8018" containerName="console" Jul 06 18:22:13.704714 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704661 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerName="config-reloader" Jul 06 18:22:13.704714 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704666 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerName="config-reloader" Jul 06 18:22:13.704714 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704677 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerName="kube-rbac-proxy-web" Jul 06 18:22:13.704714 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704682 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerName="kube-rbac-proxy-web" Jul 06 18:22:13.704714 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704689 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a635941-040e-4d5d-b8d8-6c04c6a4d2f0" containerName="console" Jul 06 18:22:13.704714 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704694 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a635941-040e-4d5d-b8d8-6c04c6a4d2f0" containerName="console" Jul 06 18:22:13.704714 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704701 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerName="kube-rbac-proxy" Jul 06 18:22:13.704714 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704705 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerName="kube-rbac-proxy" Jul 06 18:22:13.705241 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704745 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a62d81e-42fa-4604-ab20-11130d5cd553" containerName="console" Jul 06 18:22:13.705241 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704752 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerName="kube-rbac-proxy" Jul 06 18:22:13.705241 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704759 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="be5705f0-bde0-4b0f-a699-5f2b6b3d8018" containerName="console" Jul 06 18:22:13.705241 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704764 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerName="kube-rbac-proxy-metric" Jul 06 18:22:13.705241 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704770 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerName="alertmanager" Jul 06 18:22:13.705241 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704776 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerName="kube-rbac-proxy-web" Jul 06 18:22:13.705241 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704781 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerName="config-reloader" Jul 06 18:22:13.705241 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704787 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" containerName="prom-label-proxy" Jul 06 18:22:13.705241 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.704793 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a635941-040e-4d5d-b8d8-6c04c6a4d2f0" containerName="console" Jul 06 18:22:13.708829 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.708807 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.714370 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.714344 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 18:22:13.714480 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.714397 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 18:22:13.714480 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.714408 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 18:22:13.714480 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.714397 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 18:22:13.714686 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.714672 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-975lj\"" Jul 06 18:22:13.714898 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.714742 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 18:22:13.714898 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.714747 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 18:22:13.714898 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.714859 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 18:22:13.715088 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.714896 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 18:22:13.721223 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.721204 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 18:22:13.727488 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.727427 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 18:22:13.848379 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.848346 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dde180b5-8bfa-42de-b9d0-aa713d3c7855" path="/var/lib/kubelet/pods/dde180b5-8bfa-42de-b9d0-aa713d3c7855/volumes" Jul 06 18:22:13.872724 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.872698 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/82733679-085c-4388-af3e-83eafbd94a81-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.872823 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.872743 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/82733679-085c-4388-af3e-83eafbd94a81-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.872823 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.872768 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/82733679-085c-4388-af3e-83eafbd94a81-config-volume\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.872823 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.872819 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82733679-085c-4388-af3e-83eafbd94a81-web-config\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.872978 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.872836 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82733679-085c-4388-af3e-83eafbd94a81-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.872978 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.872855 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/82733679-085c-4388-af3e-83eafbd94a81-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.872978 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.872894 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82733679-085c-4388-af3e-83eafbd94a81-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.873090 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.872972 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82733679-085c-4388-af3e-83eafbd94a81-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.873090 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.873022 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82733679-085c-4388-af3e-83eafbd94a81-tls-assets\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.873090 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.873042 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82733679-085c-4388-af3e-83eafbd94a81-config-out\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.873090 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.873062 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82733679-085c-4388-af3e-83eafbd94a81-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.873090 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.873079 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jctmn\" (UniqueName: \"kubernetes.io/projected/82733679-085c-4388-af3e-83eafbd94a81-kube-api-access-jctmn\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.873242 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.873099 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/82733679-085c-4388-af3e-83eafbd94a81-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.974100 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.974046 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82733679-085c-4388-af3e-83eafbd94a81-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.974100 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.974104 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82733679-085c-4388-af3e-83eafbd94a81-tls-assets\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.974351 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.974122 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82733679-085c-4388-af3e-83eafbd94a81-config-out\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.974351 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.974140 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82733679-085c-4388-af3e-83eafbd94a81-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.974351 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.974261 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jctmn\" (UniqueName: \"kubernetes.io/projected/82733679-085c-4388-af3e-83eafbd94a81-kube-api-access-jctmn\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.974351 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.974297 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/82733679-085c-4388-af3e-83eafbd94a81-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.974351 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.974345 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/82733679-085c-4388-af3e-83eafbd94a81-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.974664 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.974399 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/82733679-085c-4388-af3e-83eafbd94a81-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.974664 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.974441 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/82733679-085c-4388-af3e-83eafbd94a81-config-volume\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.974664 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.974493 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82733679-085c-4388-af3e-83eafbd94a81-web-config\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.974664 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.974524 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82733679-085c-4388-af3e-83eafbd94a81-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.974664 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.974562 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/82733679-085c-4388-af3e-83eafbd94a81-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.974664 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.974590 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82733679-085c-4388-af3e-83eafbd94a81-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.975164 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.974907 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82733679-085c-4388-af3e-83eafbd94a81-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.976414 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.975776 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82733679-085c-4388-af3e-83eafbd94a81-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.976414 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.976042 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/82733679-085c-4388-af3e-83eafbd94a81-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.977422 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.977397 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82733679-085c-4388-af3e-83eafbd94a81-tls-assets\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.977629 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.977573 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82733679-085c-4388-af3e-83eafbd94a81-config-out\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.977724 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.977674 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/82733679-085c-4388-af3e-83eafbd94a81-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.977803 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.977787 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/82733679-085c-4388-af3e-83eafbd94a81-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.977979 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.977954 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82733679-085c-4388-af3e-83eafbd94a81-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.978081 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.978029 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82733679-085c-4388-af3e-83eafbd94a81-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.978188 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.978170 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/82733679-085c-4388-af3e-83eafbd94a81-config-volume\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.978683 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.978662 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82733679-085c-4388-af3e-83eafbd94a81-web-config\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.979611 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.979594 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/82733679-085c-4388-af3e-83eafbd94a81-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:13.983046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:13.983026 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jctmn\" (UniqueName: \"kubernetes.io/projected/82733679-085c-4388-af3e-83eafbd94a81-kube-api-access-jctmn\") pod \"alertmanager-main-0\" (UID: \"82733679-085c-4388-af3e-83eafbd94a81\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:14.018676 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:14.018655 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 18:22:14.160265 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:14.160239 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 18:22:14.162393 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:22:14.162365 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82733679_085c_4388_af3e_83eafbd94a81.slice/crio-b882c38c5b055db56a70cb9734b2659b99f262fa36e66bac1152700caf56172d WatchSource:0}: Error finding container b882c38c5b055db56a70cb9734b2659b99f262fa36e66bac1152700caf56172d: Status 404 returned error can't find the container with id b882c38c5b055db56a70cb9734b2659b99f262fa36e66bac1152700caf56172d Jul 06 18:22:14.644043 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:14.644005 2566 generic.go:356] "Generic (PLEG): container finished" podID="82733679-085c-4388-af3e-83eafbd94a81" containerID="edc9226c2d352f8996e9af89c1dad9242ec893aefbf4282f2759f96d6fce567c" exitCode=0 Jul 06 18:22:14.644486 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:14.644078 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82733679-085c-4388-af3e-83eafbd94a81","Type":"ContainerDied","Data":"edc9226c2d352f8996e9af89c1dad9242ec893aefbf4282f2759f96d6fce567c"} Jul 06 18:22:14.644486 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:14.644153 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82733679-085c-4388-af3e-83eafbd94a81","Type":"ContainerStarted","Data":"b882c38c5b055db56a70cb9734b2659b99f262fa36e66bac1152700caf56172d"} Jul 06 18:22:15.653408 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:15.653301 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82733679-085c-4388-af3e-83eafbd94a81","Type":"ContainerStarted","Data":"d0dd5a2f84cd2f4f2c9d137659afd719ba237f5462aab5e21f5067b632c874a8"} Jul 06 18:22:15.654058 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:15.653507 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82733679-085c-4388-af3e-83eafbd94a81","Type":"ContainerStarted","Data":"66d1cd528b901b9ac37aadad2d74cdb5cebdf85638c30d8ef3f60910f7559c2c"} Jul 06 18:22:15.654058 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:15.653550 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82733679-085c-4388-af3e-83eafbd94a81","Type":"ContainerStarted","Data":"953ea7e4d0830f9d14c376c5ac521cc641f20bf64ac237ea08ed808a676b86c6"} Jul 06 18:22:15.654058 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:15.653588 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82733679-085c-4388-af3e-83eafbd94a81","Type":"ContainerStarted","Data":"f25ffc93486aa85d6388a63a5be401090a2c5ebb7580c41b94a6280a2be7f538"} Jul 06 18:22:15.654058 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:15.653627 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82733679-085c-4388-af3e-83eafbd94a81","Type":"ContainerStarted","Data":"c6d66b8c7580537e9a1ae7c386d99e3dc2980eba23239e5b501db10ad9b9275c"} Jul 06 18:22:15.654058 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:15.653667 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82733679-085c-4388-af3e-83eafbd94a81","Type":"ContainerStarted","Data":"7737708eba0b21bc4b3dab9c69f9edd98000df6f15237865a0248c113c9df33f"} Jul 06 18:22:15.686990 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:15.686920 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.67120182 podStartE2EDuration="2.686897687s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.695867ms" isStatefulPod=false podCreationTimestamp="2026-07-06 18:22:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:22:15.684549345 +0000 UTC m=+152.410978333" watchObservedRunningTime="2026-07-06 18:22:15.686897687 +0000 UTC m=+152.413326648" Jul 06 18:22:17.116718 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.116687 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58b479586d-rf8wx"] Jul 06 18:22:17.119177 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.119160 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58b479586d-rf8wx" Jul 06 18:22:17.143817 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.143785 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58b479586d-rf8wx"] Jul 06 18:22:17.184846 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.184816 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:22:17.261144 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.261114 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:22:17.304648 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.304608 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7ad4e548-2868-43ec-b0fc-dae0d329ab18-console-oauth-config\") pod \"console-58b479586d-rf8wx\" (UID: \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\") " pod="openshift-console/console-58b479586d-rf8wx" Jul 06 18:22:17.304648 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.304645 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ss7m\" (UniqueName: \"kubernetes.io/projected/7ad4e548-2868-43ec-b0fc-dae0d329ab18-kube-api-access-9ss7m\") pod \"console-58b479586d-rf8wx\" (UID: \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\") " pod="openshift-console/console-58b479586d-rf8wx" Jul 06 18:22:17.304901 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.304676 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7ad4e548-2868-43ec-b0fc-dae0d329ab18-console-serving-cert\") pod \"console-58b479586d-rf8wx\" (UID: \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\") " pod="openshift-console/console-58b479586d-rf8wx" Jul 06 18:22:17.304901 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.304718 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7ad4e548-2868-43ec-b0fc-dae0d329ab18-service-ca\") pod \"console-58b479586d-rf8wx\" (UID: \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\") " pod="openshift-console/console-58b479586d-rf8wx" Jul 06 18:22:17.304901 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.304747 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7ad4e548-2868-43ec-b0fc-dae0d329ab18-oauth-serving-cert\") pod \"console-58b479586d-rf8wx\" (UID: \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\") " pod="openshift-console/console-58b479586d-rf8wx" Jul 06 18:22:17.304901 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.304769 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ad4e548-2868-43ec-b0fc-dae0d329ab18-trusted-ca-bundle\") pod \"console-58b479586d-rf8wx\" (UID: \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\") " pod="openshift-console/console-58b479586d-rf8wx" Jul 06 18:22:17.304901 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.304854 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7ad4e548-2868-43ec-b0fc-dae0d329ab18-console-config\") pod \"console-58b479586d-rf8wx\" (UID: \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\") " pod="openshift-console/console-58b479586d-rf8wx" Jul 06 18:22:17.348095 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.348060 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:22:17.406064 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.406038 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7ad4e548-2868-43ec-b0fc-dae0d329ab18-console-config\") pod \"console-58b479586d-rf8wx\" (UID: \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\") " pod="openshift-console/console-58b479586d-rf8wx" Jul 06 18:22:17.406234 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.406083 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7ad4e548-2868-43ec-b0fc-dae0d329ab18-console-oauth-config\") pod \"console-58b479586d-rf8wx\" (UID: \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\") " pod="openshift-console/console-58b479586d-rf8wx" Jul 06 18:22:17.406234 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.406109 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ss7m\" (UniqueName: \"kubernetes.io/projected/7ad4e548-2868-43ec-b0fc-dae0d329ab18-kube-api-access-9ss7m\") pod \"console-58b479586d-rf8wx\" (UID: \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\") " pod="openshift-console/console-58b479586d-rf8wx" Jul 06 18:22:17.406234 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.406133 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7ad4e548-2868-43ec-b0fc-dae0d329ab18-console-serving-cert\") pod \"console-58b479586d-rf8wx\" (UID: \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\") " pod="openshift-console/console-58b479586d-rf8wx" Jul 06 18:22:17.406353 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.406275 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7ad4e548-2868-43ec-b0fc-dae0d329ab18-service-ca\") pod \"console-58b479586d-rf8wx\" (UID: \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\") " pod="openshift-console/console-58b479586d-rf8wx" Jul 06 18:22:17.406353 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.406334 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7ad4e548-2868-43ec-b0fc-dae0d329ab18-oauth-serving-cert\") pod \"console-58b479586d-rf8wx\" (UID: \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\") " pod="openshift-console/console-58b479586d-rf8wx" Jul 06 18:22:17.406474 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.406369 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ad4e548-2868-43ec-b0fc-dae0d329ab18-trusted-ca-bundle\") pod \"console-58b479586d-rf8wx\" (UID: \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\") " pod="openshift-console/console-58b479586d-rf8wx" Jul 06 18:22:17.407028 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.407003 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7ad4e548-2868-43ec-b0fc-dae0d329ab18-oauth-serving-cert\") pod \"console-58b479586d-rf8wx\" (UID: \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\") " pod="openshift-console/console-58b479586d-rf8wx" Jul 06 18:22:17.407168 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.407130 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7ad4e548-2868-43ec-b0fc-dae0d329ab18-console-config\") pod \"console-58b479586d-rf8wx\" (UID: \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\") " pod="openshift-console/console-58b479586d-rf8wx" Jul 06 18:22:17.407234 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.407191 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ad4e548-2868-43ec-b0fc-dae0d329ab18-trusted-ca-bundle\") pod \"console-58b479586d-rf8wx\" (UID: \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\") " pod="openshift-console/console-58b479586d-rf8wx" Jul 06 18:22:17.407234 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.407217 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7ad4e548-2868-43ec-b0fc-dae0d329ab18-service-ca\") pod \"console-58b479586d-rf8wx\" (UID: \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\") " pod="openshift-console/console-58b479586d-rf8wx" Jul 06 18:22:17.408718 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.408684 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7ad4e548-2868-43ec-b0fc-dae0d329ab18-console-serving-cert\") pod \"console-58b479586d-rf8wx\" (UID: \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\") " pod="openshift-console/console-58b479586d-rf8wx" Jul 06 18:22:17.408837 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.408817 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7ad4e548-2868-43ec-b0fc-dae0d329ab18-console-oauth-config\") pod \"console-58b479586d-rf8wx\" (UID: \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\") " pod="openshift-console/console-58b479586d-rf8wx" Jul 06 18:22:17.414401 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.414382 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ss7m\" (UniqueName: \"kubernetes.io/projected/7ad4e548-2868-43ec-b0fc-dae0d329ab18-kube-api-access-9ss7m\") pod \"console-58b479586d-rf8wx\" (UID: \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\") " pod="openshift-console/console-58b479586d-rf8wx" Jul 06 18:22:17.428475 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.428445 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58b479586d-rf8wx" Jul 06 18:22:17.561988 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.561962 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58b479586d-rf8wx"] Jul 06 18:22:17.564193 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:22:17.564168 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ad4e548_2868_43ec_b0fc_dae0d329ab18.slice/crio-6e087ed58ae7ee22111b4c60d8109f82990339edfef5688cbeb43128fa08dfc0 WatchSource:0}: Error finding container 6e087ed58ae7ee22111b4c60d8109f82990339edfef5688cbeb43128fa08dfc0: Status 404 returned error can't find the container with id 6e087ed58ae7ee22111b4c60d8109f82990339edfef5688cbeb43128fa08dfc0 Jul 06 18:22:17.664960 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.664846 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58b479586d-rf8wx" event={"ID":"7ad4e548-2868-43ec-b0fc-dae0d329ab18","Type":"ContainerStarted","Data":"ad2d8a83e03c42af245dbb178181e50a247bb2975ea2e14d919295660aa65b45"} Jul 06 18:22:17.664960 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.664931 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58b479586d-rf8wx" event={"ID":"7ad4e548-2868-43ec-b0fc-dae0d329ab18","Type":"ContainerStarted","Data":"6e087ed58ae7ee22111b4c60d8109f82990339edfef5688cbeb43128fa08dfc0"} Jul 06 18:22:17.684194 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.684143 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58b479586d-rf8wx" podStartSLOduration=0.684127196 podStartE2EDuration="684.127196ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:22:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:22:17.683163644 +0000 UTC m=+154.409592616" watchObservedRunningTime="2026-07-06 18:22:17.684127196 +0000 UTC m=+154.410556153" Jul 06 18:22:17.861095 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.861061 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:22:17.922733 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:17.922657 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:22:19.056395 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:19.056365 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:22:19.111707 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:19.111676 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:22:25.142698 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:25.142666 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:22:25.243627 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:25.243596 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:22:27.429025 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:27.428977 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-58b479586d-rf8wx" Jul 06 18:22:27.429025 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:27.429044 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58b479586d-rf8wx" Jul 06 18:22:27.433715 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:27.433692 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-58b479586d-rf8wx" Jul 06 18:22:27.709701 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:27.709624 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-58b479586d-rf8wx" Jul 06 18:22:27.758215 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:27.758187 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-586c9fc948-wjphf"] Jul 06 18:22:27.859081 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:27.859048 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:22:27.959533 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:27.959502 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:22:28.027861 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:28.027788 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:22:37.650710 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:37.650631 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:22:52.784814 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:52.784766 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-586c9fc948-wjphf" podUID="b00fa2f2-9886-4c5a-9970-04465cea42dd" containerName="console" containerID="cri-o://51d9a6f5c0cbc81cb715bafb8d1631648b8d1a1997ed221617f24691acbe0d32" gracePeriod=15 Jul 06 18:22:53.030409 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.030382 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-586c9fc948-wjphf" Jul 06 18:22:53.100582 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.100497 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b00fa2f2-9886-4c5a-9970-04465cea42dd-oauth-serving-cert\") pod \"b00fa2f2-9886-4c5a-9970-04465cea42dd\" (UID: \"b00fa2f2-9886-4c5a-9970-04465cea42dd\") " Jul 06 18:22:53.100582 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.100529 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pc42\" (UniqueName: \"kubernetes.io/projected/b00fa2f2-9886-4c5a-9970-04465cea42dd-kube-api-access-8pc42\") pod \"b00fa2f2-9886-4c5a-9970-04465cea42dd\" (UID: \"b00fa2f2-9886-4c5a-9970-04465cea42dd\") " Jul 06 18:22:53.100582 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.100545 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b00fa2f2-9886-4c5a-9970-04465cea42dd-trusted-ca-bundle\") pod \"b00fa2f2-9886-4c5a-9970-04465cea42dd\" (UID: \"b00fa2f2-9886-4c5a-9970-04465cea42dd\") " Jul 06 18:22:53.100582 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.100574 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b00fa2f2-9886-4c5a-9970-04465cea42dd-console-oauth-config\") pod \"b00fa2f2-9886-4c5a-9970-04465cea42dd\" (UID: \"b00fa2f2-9886-4c5a-9970-04465cea42dd\") " Jul 06 18:22:53.100901 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.100752 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b00fa2f2-9886-4c5a-9970-04465cea42dd-console-serving-cert\") pod \"b00fa2f2-9886-4c5a-9970-04465cea42dd\" (UID: \"b00fa2f2-9886-4c5a-9970-04465cea42dd\") " Jul 06 18:22:53.100901 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.100805 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b00fa2f2-9886-4c5a-9970-04465cea42dd-service-ca\") pod \"b00fa2f2-9886-4c5a-9970-04465cea42dd\" (UID: \"b00fa2f2-9886-4c5a-9970-04465cea42dd\") " Jul 06 18:22:53.101025 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.100923 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b00fa2f2-9886-4c5a-9970-04465cea42dd-console-config\") pod \"b00fa2f2-9886-4c5a-9970-04465cea42dd\" (UID: \"b00fa2f2-9886-4c5a-9970-04465cea42dd\") " Jul 06 18:22:53.101073 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.101008 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b00fa2f2-9886-4c5a-9970-04465cea42dd-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b00fa2f2-9886-4c5a-9970-04465cea42dd" (UID: "b00fa2f2-9886-4c5a-9970-04465cea42dd"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:22:53.101152 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.101128 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b00fa2f2-9886-4c5a-9970-04465cea42dd-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b00fa2f2-9886-4c5a-9970-04465cea42dd" (UID: "b00fa2f2-9886-4c5a-9970-04465cea42dd"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:22:53.101224 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.101197 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b00fa2f2-9886-4c5a-9970-04465cea42dd-service-ca" (OuterVolumeSpecName: "service-ca") pod "b00fa2f2-9886-4c5a-9970-04465cea42dd" (UID: "b00fa2f2-9886-4c5a-9970-04465cea42dd"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:22:53.101282 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.101213 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b00fa2f2-9886-4c5a-9970-04465cea42dd-oauth-serving-cert\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:22:53.101282 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.101251 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b00fa2f2-9886-4c5a-9970-04465cea42dd-console-config" (OuterVolumeSpecName: "console-config") pod "b00fa2f2-9886-4c5a-9970-04465cea42dd" (UID: "b00fa2f2-9886-4c5a-9970-04465cea42dd"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:22:53.101282 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.101259 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b00fa2f2-9886-4c5a-9970-04465cea42dd-trusted-ca-bundle\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:22:53.102819 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.102797 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b00fa2f2-9886-4c5a-9970-04465cea42dd-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b00fa2f2-9886-4c5a-9970-04465cea42dd" (UID: "b00fa2f2-9886-4c5a-9970-04465cea42dd"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:22:53.102917 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.102893 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b00fa2f2-9886-4c5a-9970-04465cea42dd-kube-api-access-8pc42" (OuterVolumeSpecName: "kube-api-access-8pc42") pod "b00fa2f2-9886-4c5a-9970-04465cea42dd" (UID: "b00fa2f2-9886-4c5a-9970-04465cea42dd"). InnerVolumeSpecName "kube-api-access-8pc42". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:22:53.102966 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.102949 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b00fa2f2-9886-4c5a-9970-04465cea42dd-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b00fa2f2-9886-4c5a-9970-04465cea42dd" (UID: "b00fa2f2-9886-4c5a-9970-04465cea42dd"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:22:53.202498 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.202463 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b00fa2f2-9886-4c5a-9970-04465cea42dd-console-config\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:22:53.202498 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.202493 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pc42\" (UniqueName: \"kubernetes.io/projected/b00fa2f2-9886-4c5a-9970-04465cea42dd-kube-api-access-8pc42\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:22:53.202498 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.202503 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b00fa2f2-9886-4c5a-9970-04465cea42dd-console-oauth-config\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:22:53.202498 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.202511 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b00fa2f2-9886-4c5a-9970-04465cea42dd-console-serving-cert\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:22:53.202769 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.202520 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b00fa2f2-9886-4c5a-9970-04465cea42dd-service-ca\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:22:53.807582 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.807541 2566 generic.go:356] "Generic (PLEG): container finished" podID="b00fa2f2-9886-4c5a-9970-04465cea42dd" containerID="51d9a6f5c0cbc81cb715bafb8d1631648b8d1a1997ed221617f24691acbe0d32" exitCode=0 Jul 06 18:22:53.808072 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.807587 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-586c9fc948-wjphf" event={"ID":"b00fa2f2-9886-4c5a-9970-04465cea42dd","Type":"ContainerDied","Data":"51d9a6f5c0cbc81cb715bafb8d1631648b8d1a1997ed221617f24691acbe0d32"} Jul 06 18:22:53.808072 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.807616 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-586c9fc948-wjphf" Jul 06 18:22:53.808072 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.807653 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-586c9fc948-wjphf" event={"ID":"b00fa2f2-9886-4c5a-9970-04465cea42dd","Type":"ContainerDied","Data":"5d38c652f4fe3bb7196ee32b8375e6b4283075db2725e59b4d4b6e36fda63ab0"} Jul 06 18:22:53.808072 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.807683 2566 scope.go:117] "RemoveContainer" containerID="51d9a6f5c0cbc81cb715bafb8d1631648b8d1a1997ed221617f24691acbe0d32" Jul 06 18:22:53.816363 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.816347 2566 scope.go:117] "RemoveContainer" containerID="51d9a6f5c0cbc81cb715bafb8d1631648b8d1a1997ed221617f24691acbe0d32" Jul 06 18:22:53.816609 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:22:53.816590 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51d9a6f5c0cbc81cb715bafb8d1631648b8d1a1997ed221617f24691acbe0d32\": container with ID starting with 51d9a6f5c0cbc81cb715bafb8d1631648b8d1a1997ed221617f24691acbe0d32 not found: ID does not exist" containerID="51d9a6f5c0cbc81cb715bafb8d1631648b8d1a1997ed221617f24691acbe0d32" Jul 06 18:22:53.816672 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.816619 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51d9a6f5c0cbc81cb715bafb8d1631648b8d1a1997ed221617f24691acbe0d32"} err="failed to get container status \"51d9a6f5c0cbc81cb715bafb8d1631648b8d1a1997ed221617f24691acbe0d32\": rpc error: code = NotFound desc = could not find container \"51d9a6f5c0cbc81cb715bafb8d1631648b8d1a1997ed221617f24691acbe0d32\": container with ID starting with 51d9a6f5c0cbc81cb715bafb8d1631648b8d1a1997ed221617f24691acbe0d32 not found: ID does not exist" Jul 06 18:22:53.827757 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.827733 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-586c9fc948-wjphf"] Jul 06 18:22:53.835951 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.832832 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-586c9fc948-wjphf"] Jul 06 18:22:53.849333 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:53.849302 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b00fa2f2-9886-4c5a-9970-04465cea42dd" path="/var/lib/kubelet/pods/b00fa2f2-9886-4c5a-9970-04465cea42dd/volumes" Jul 06 18:22:59.175857 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:22:59.175811 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:23:10.071131 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:10.071098 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:23:10.121301 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:10.121263 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:23:16.046507 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:16.046465 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:23:16.709179 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:16.709141 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:23:16.809004 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:16.808972 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:23:17.380135 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:17.380104 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44"] Jul 06 18:23:17.380491 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:17.380432 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b00fa2f2-9886-4c5a-9970-04465cea42dd" containerName="console" Jul 06 18:23:17.380491 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:17.380443 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b00fa2f2-9886-4c5a-9970-04465cea42dd" containerName="console" Jul 06 18:23:17.380559 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:17.380508 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b00fa2f2-9886-4c5a-9970-04465cea42dd" containerName="console" Jul 06 18:23:17.382858 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:17.382840 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44" Jul 06 18:23:17.386117 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:17.386087 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rqp65\"" Jul 06 18:23:17.386221 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:17.386124 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 18:23:17.387345 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:17.387329 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 18:23:17.393216 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:17.393196 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44"] Jul 06 18:23:17.518318 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:17.518284 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhnfk\" (UniqueName: \"kubernetes.io/projected/47353d18-6021-46c1-a0f7-802f5d6182e3-kube-api-access-lhnfk\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44\" (UID: \"47353d18-6021-46c1-a0f7-802f5d6182e3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44" Jul 06 18:23:17.518512 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:17.518342 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47353d18-6021-46c1-a0f7-802f5d6182e3-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44\" (UID: \"47353d18-6021-46c1-a0f7-802f5d6182e3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44" Jul 06 18:23:17.518512 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:17.518364 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47353d18-6021-46c1-a0f7-802f5d6182e3-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44\" (UID: \"47353d18-6021-46c1-a0f7-802f5d6182e3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44" Jul 06 18:23:17.618774 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:17.618735 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47353d18-6021-46c1-a0f7-802f5d6182e3-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44\" (UID: \"47353d18-6021-46c1-a0f7-802f5d6182e3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44" Jul 06 18:23:17.618774 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:17.618777 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47353d18-6021-46c1-a0f7-802f5d6182e3-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44\" (UID: \"47353d18-6021-46c1-a0f7-802f5d6182e3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44" Jul 06 18:23:17.619033 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:17.618850 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhnfk\" (UniqueName: \"kubernetes.io/projected/47353d18-6021-46c1-a0f7-802f5d6182e3-kube-api-access-lhnfk\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44\" (UID: \"47353d18-6021-46c1-a0f7-802f5d6182e3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44" Jul 06 18:23:17.619134 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:17.619114 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47353d18-6021-46c1-a0f7-802f5d6182e3-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44\" (UID: \"47353d18-6021-46c1-a0f7-802f5d6182e3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44" Jul 06 18:23:17.619191 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:17.619145 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47353d18-6021-46c1-a0f7-802f5d6182e3-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44\" (UID: \"47353d18-6021-46c1-a0f7-802f5d6182e3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44" Jul 06 18:23:17.629132 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:17.629109 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhnfk\" (UniqueName: \"kubernetes.io/projected/47353d18-6021-46c1-a0f7-802f5d6182e3-kube-api-access-lhnfk\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44\" (UID: \"47353d18-6021-46c1-a0f7-802f5d6182e3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44" Jul 06 18:23:17.692496 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:17.692455 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44" Jul 06 18:23:17.821973 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:17.821947 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44"] Jul 06 18:23:17.824691 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:23:17.824660 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47353d18_6021_46c1_a0f7_802f5d6182e3.slice/crio-bce37908681dd9392cbf1e4078c70592fc4908a85e9b2d3a9499c8eff2eafc51 WatchSource:0}: Error finding container bce37908681dd9392cbf1e4078c70592fc4908a85e9b2d3a9499c8eff2eafc51: Status 404 returned error can't find the container with id bce37908681dd9392cbf1e4078c70592fc4908a85e9b2d3a9499c8eff2eafc51 Jul 06 18:23:17.900929 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:17.900895 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44" event={"ID":"47353d18-6021-46c1-a0f7-802f5d6182e3","Type":"ContainerStarted","Data":"bce37908681dd9392cbf1e4078c70592fc4908a85e9b2d3a9499c8eff2eafc51"} Jul 06 18:23:24.932635 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:24.932596 2566 generic.go:356] "Generic (PLEG): container finished" podID="47353d18-6021-46c1-a0f7-802f5d6182e3" containerID="38cb8858b214fd1c6675f9344145987df760be88fddbdf3eed6a255b28564c0d" exitCode=0 Jul 06 18:23:24.933042 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:24.932687 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44" event={"ID":"47353d18-6021-46c1-a0f7-802f5d6182e3","Type":"ContainerDied","Data":"38cb8858b214fd1c6675f9344145987df760be88fddbdf3eed6a255b28564c0d"} Jul 06 18:23:27.947044 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:27.947011 2566 generic.go:356] "Generic (PLEG): container finished" podID="47353d18-6021-46c1-a0f7-802f5d6182e3" containerID="b3c1400896e496ee0854b422d180efe197aba66f355083fbedb874fb3c37a9ad" exitCode=0 Jul 06 18:23:27.947439 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:27.947097 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44" event={"ID":"47353d18-6021-46c1-a0f7-802f5d6182e3","Type":"ContainerDied","Data":"b3c1400896e496ee0854b422d180efe197aba66f355083fbedb874fb3c37a9ad"} Jul 06 18:23:36.990341 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:36.990305 2566 generic.go:356] "Generic (PLEG): container finished" podID="47353d18-6021-46c1-a0f7-802f5d6182e3" containerID="37c78462b33059cc20a7b7e56e99c9ac2fd6df2d1e8031b52925e1e25a94173b" exitCode=0 Jul 06 18:23:36.990738 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:36.990381 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44" event={"ID":"47353d18-6021-46c1-a0f7-802f5d6182e3","Type":"ContainerDied","Data":"37c78462b33059cc20a7b7e56e99c9ac2fd6df2d1e8031b52925e1e25a94173b"} Jul 06 18:23:38.116523 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:38.116494 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44" Jul 06 18:23:38.189142 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:38.189106 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47353d18-6021-46c1-a0f7-802f5d6182e3-util\") pod \"47353d18-6021-46c1-a0f7-802f5d6182e3\" (UID: \"47353d18-6021-46c1-a0f7-802f5d6182e3\") " Jul 06 18:23:38.189333 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:38.189168 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47353d18-6021-46c1-a0f7-802f5d6182e3-bundle\") pod \"47353d18-6021-46c1-a0f7-802f5d6182e3\" (UID: \"47353d18-6021-46c1-a0f7-802f5d6182e3\") " Jul 06 18:23:38.189333 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:38.189206 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhnfk\" (UniqueName: \"kubernetes.io/projected/47353d18-6021-46c1-a0f7-802f5d6182e3-kube-api-access-lhnfk\") pod \"47353d18-6021-46c1-a0f7-802f5d6182e3\" (UID: \"47353d18-6021-46c1-a0f7-802f5d6182e3\") " Jul 06 18:23:38.189883 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:38.189839 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47353d18-6021-46c1-a0f7-802f5d6182e3-bundle" (OuterVolumeSpecName: "bundle") pod "47353d18-6021-46c1-a0f7-802f5d6182e3" (UID: "47353d18-6021-46c1-a0f7-802f5d6182e3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:23:38.191575 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:38.191541 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47353d18-6021-46c1-a0f7-802f5d6182e3-kube-api-access-lhnfk" (OuterVolumeSpecName: "kube-api-access-lhnfk") pod "47353d18-6021-46c1-a0f7-802f5d6182e3" (UID: "47353d18-6021-46c1-a0f7-802f5d6182e3"). InnerVolumeSpecName "kube-api-access-lhnfk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:23:38.193659 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:38.193638 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47353d18-6021-46c1-a0f7-802f5d6182e3-util" (OuterVolumeSpecName: "util") pod "47353d18-6021-46c1-a0f7-802f5d6182e3" (UID: "47353d18-6021-46c1-a0f7-802f5d6182e3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:23:38.290010 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:38.289925 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47353d18-6021-46c1-a0f7-802f5d6182e3-util\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:23:38.290010 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:38.289953 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47353d18-6021-46c1-a0f7-802f5d6182e3-bundle\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:23:38.290010 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:38.289963 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhnfk\" (UniqueName: \"kubernetes.io/projected/47353d18-6021-46c1-a0f7-802f5d6182e3-kube-api-access-lhnfk\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:23:39.000425 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:39.000392 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44" Jul 06 18:23:39.000608 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:39.000426 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j4p44" event={"ID":"47353d18-6021-46c1-a0f7-802f5d6182e3","Type":"ContainerDied","Data":"bce37908681dd9392cbf1e4078c70592fc4908a85e9b2d3a9499c8eff2eafc51"} Jul 06 18:23:39.000608 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:39.000479 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bce37908681dd9392cbf1e4078c70592fc4908a85e9b2d3a9499c8eff2eafc51" Jul 06 18:23:43.448987 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:43.448948 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:23:43.550940 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:43.550909 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:23:44.236608 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:44.236567 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:23:45.348816 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:45.348779 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m722g"] Jul 06 18:23:45.349349 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:45.349258 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47353d18-6021-46c1-a0f7-802f5d6182e3" containerName="util" Jul 06 18:23:45.349349 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:45.349277 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="47353d18-6021-46c1-a0f7-802f5d6182e3" containerName="util" Jul 06 18:23:45.349349 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:45.349290 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47353d18-6021-46c1-a0f7-802f5d6182e3" containerName="pull" Jul 06 18:23:45.349349 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:45.349298 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="47353d18-6021-46c1-a0f7-802f5d6182e3" containerName="pull" Jul 06 18:23:45.349349 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:45.349321 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47353d18-6021-46c1-a0f7-802f5d6182e3" containerName="extract" Jul 06 18:23:45.349349 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:45.349328 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="47353d18-6021-46c1-a0f7-802f5d6182e3" containerName="extract" Jul 06 18:23:45.349644 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:45.349402 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="47353d18-6021-46c1-a0f7-802f5d6182e3" containerName="extract" Jul 06 18:23:45.353198 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:45.353174 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m722g" Jul 06 18:23:45.356385 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:45.356362 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-wxq4b\"" Jul 06 18:23:45.356798 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:45.356776 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 06 18:23:45.356943 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:45.356782 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 06 18:23:45.368933 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:45.368906 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m722g"] Jul 06 18:23:45.457156 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:45.457119 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8379960f-785c-4ecc-95fa-7a6e12c772d9-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-m722g\" (UID: \"8379960f-785c-4ecc-95fa-7a6e12c772d9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m722g" Jul 06 18:23:45.457338 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:45.457212 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8twx\" (UniqueName: \"kubernetes.io/projected/8379960f-785c-4ecc-95fa-7a6e12c772d9-kube-api-access-m8twx\") pod \"cert-manager-operator-controller-manager-55c444bf86-m722g\" (UID: \"8379960f-785c-4ecc-95fa-7a6e12c772d9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m722g" Jul 06 18:23:45.558561 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:45.558528 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8379960f-785c-4ecc-95fa-7a6e12c772d9-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-m722g\" (UID: \"8379960f-785c-4ecc-95fa-7a6e12c772d9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m722g" Jul 06 18:23:45.558759 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:45.558595 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8twx\" (UniqueName: \"kubernetes.io/projected/8379960f-785c-4ecc-95fa-7a6e12c772d9-kube-api-access-m8twx\") pod \"cert-manager-operator-controller-manager-55c444bf86-m722g\" (UID: \"8379960f-785c-4ecc-95fa-7a6e12c772d9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m722g" Jul 06 18:23:45.558928 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:45.558906 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8379960f-785c-4ecc-95fa-7a6e12c772d9-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-m722g\" (UID: \"8379960f-785c-4ecc-95fa-7a6e12c772d9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m722g" Jul 06 18:23:45.568088 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:45.568057 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8twx\" (UniqueName: \"kubernetes.io/projected/8379960f-785c-4ecc-95fa-7a6e12c772d9-kube-api-access-m8twx\") pod \"cert-manager-operator-controller-manager-55c444bf86-m722g\" (UID: \"8379960f-785c-4ecc-95fa-7a6e12c772d9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m722g" Jul 06 18:23:45.662942 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:45.662906 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m722g" Jul 06 18:23:45.806764 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:45.806561 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m722g"] Jul 06 18:23:45.808345 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:23:45.808316 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8379960f_785c_4ecc_95fa_7a6e12c772d9.slice/crio-30e4a2553666071d56badab8efc211478c41deb29841dc1d035293329ac21214 WatchSource:0}: Error finding container 30e4a2553666071d56badab8efc211478c41deb29841dc1d035293329ac21214: Status 404 returned error can't find the container with id 30e4a2553666071d56badab8efc211478c41deb29841dc1d035293329ac21214 Jul 06 18:23:46.029638 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:46.029550 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m722g" event={"ID":"8379960f-785c-4ecc-95fa-7a6e12c772d9","Type":"ContainerStarted","Data":"30e4a2553666071d56badab8efc211478c41deb29841dc1d035293329ac21214"} Jul 06 18:23:47.043326 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:47.043291 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:23:47.142102 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:47.142070 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:23:48.043324 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:48.043283 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m722g" event={"ID":"8379960f-785c-4ecc-95fa-7a6e12c772d9","Type":"ContainerStarted","Data":"9c9cfcb5954dec02602aba892915330ccd4d21ae1a7f53e6a5ae37e4d1cbcdb1"} Jul 06 18:23:48.070075 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:48.069095 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m722g" podStartSLOduration=1.479886777 podStartE2EDuration="3.069071296s" totalImagesPullingTime="1.589184519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:23:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:23:48.067332897 +0000 UTC m=+244.793761869" watchObservedRunningTime="2026-07-06 18:23:48.069071296 +0000 UTC m=+244.795500257" Jul 06 18:23:48.492006 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:48.491971 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:23:48.610302 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:48.610274 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:23:49.082307 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.082269 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp"] Jul 06 18:23:49.084964 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.084945 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp" Jul 06 18:23:49.088132 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.088106 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 18:23:49.088312 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.088296 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rqp65\"" Jul 06 18:23:49.088369 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.088354 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 18:23:49.109027 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.108996 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp"] Jul 06 18:23:49.132113 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.132080 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:23:49.193895 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.193832 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f32c5d84-d81b-4a10-a102-0fd3a939fa75-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp\" (UID: \"f32c5d84-d81b-4a10-a102-0fd3a939fa75\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp" Jul 06 18:23:49.193895 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.193894 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f32c5d84-d81b-4a10-a102-0fd3a939fa75-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp\" (UID: \"f32c5d84-d81b-4a10-a102-0fd3a939fa75\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp" Jul 06 18:23:49.194151 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.193993 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk52b\" (UniqueName: \"kubernetes.io/projected/f32c5d84-d81b-4a10-a102-0fd3a939fa75-kube-api-access-qk52b\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp\" (UID: \"f32c5d84-d81b-4a10-a102-0fd3a939fa75\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp" Jul 06 18:23:49.229913 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.229858 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:23:49.295505 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.295471 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f32c5d84-d81b-4a10-a102-0fd3a939fa75-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp\" (UID: \"f32c5d84-d81b-4a10-a102-0fd3a939fa75\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp" Jul 06 18:23:49.295505 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.295504 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f32c5d84-d81b-4a10-a102-0fd3a939fa75-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp\" (UID: \"f32c5d84-d81b-4a10-a102-0fd3a939fa75\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp" Jul 06 18:23:49.295706 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.295539 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk52b\" (UniqueName: \"kubernetes.io/projected/f32c5d84-d81b-4a10-a102-0fd3a939fa75-kube-api-access-qk52b\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp\" (UID: \"f32c5d84-d81b-4a10-a102-0fd3a939fa75\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp" Jul 06 18:23:49.295993 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.295970 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f32c5d84-d81b-4a10-a102-0fd3a939fa75-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp\" (UID: \"f32c5d84-d81b-4a10-a102-0fd3a939fa75\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp" Jul 06 18:23:49.296041 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.295979 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f32c5d84-d81b-4a10-a102-0fd3a939fa75-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp\" (UID: \"f32c5d84-d81b-4a10-a102-0fd3a939fa75\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp" Jul 06 18:23:49.305524 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.305497 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk52b\" (UniqueName: \"kubernetes.io/projected/f32c5d84-d81b-4a10-a102-0fd3a939fa75-kube-api-access-qk52b\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp\" (UID: \"f32c5d84-d81b-4a10-a102-0fd3a939fa75\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp" Jul 06 18:23:49.395452 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.395419 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp" Jul 06 18:23:49.422079 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.422042 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-md9ft"] Jul 06 18:23:49.424535 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.424515 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-md9ft" Jul 06 18:23:49.427961 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.427940 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 18:23:49.428101 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.427940 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 18:23:49.428101 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.427944 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-lflm9\"" Jul 06 18:23:49.434763 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.434736 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-md9ft"] Jul 06 18:23:49.497491 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.497456 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkcnv\" (UniqueName: \"kubernetes.io/projected/8261c8d6-57aa-4e80-a01a-b8a33e75f1f2-kube-api-access-qkcnv\") pod \"cert-manager-webhook-6486ffd997-md9ft\" (UID: \"8261c8d6-57aa-4e80-a01a-b8a33e75f1f2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-md9ft" Jul 06 18:23:49.497645 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.497518 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8261c8d6-57aa-4e80-a01a-b8a33e75f1f2-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-md9ft\" (UID: \"8261c8d6-57aa-4e80-a01a-b8a33e75f1f2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-md9ft" Jul 06 18:23:49.538795 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.538671 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp"] Jul 06 18:23:49.541413 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:23:49.541384 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf32c5d84_d81b_4a10_a102_0fd3a939fa75.slice/crio-1a7c06e87e524dd1b86962c67828026c8a0b14fc9c8e22a651a4263a5016bfa2 WatchSource:0}: Error finding container 1a7c06e87e524dd1b86962c67828026c8a0b14fc9c8e22a651a4263a5016bfa2: Status 404 returned error can't find the container with id 1a7c06e87e524dd1b86962c67828026c8a0b14fc9c8e22a651a4263a5016bfa2 Jul 06 18:23:49.598922 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.598897 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkcnv\" (UniqueName: \"kubernetes.io/projected/8261c8d6-57aa-4e80-a01a-b8a33e75f1f2-kube-api-access-qkcnv\") pod \"cert-manager-webhook-6486ffd997-md9ft\" (UID: \"8261c8d6-57aa-4e80-a01a-b8a33e75f1f2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-md9ft" Jul 06 18:23:49.599065 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.598955 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8261c8d6-57aa-4e80-a01a-b8a33e75f1f2-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-md9ft\" (UID: \"8261c8d6-57aa-4e80-a01a-b8a33e75f1f2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-md9ft" Jul 06 18:23:49.608121 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.608100 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkcnv\" (UniqueName: \"kubernetes.io/projected/8261c8d6-57aa-4e80-a01a-b8a33e75f1f2-kube-api-access-qkcnv\") pod \"cert-manager-webhook-6486ffd997-md9ft\" (UID: \"8261c8d6-57aa-4e80-a01a-b8a33e75f1f2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-md9ft" Jul 06 18:23:49.608466 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.608446 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8261c8d6-57aa-4e80-a01a-b8a33e75f1f2-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-md9ft\" (UID: \"8261c8d6-57aa-4e80-a01a-b8a33e75f1f2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-md9ft" Jul 06 18:23:49.742795 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.742753 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-md9ft" Jul 06 18:23:49.876703 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:49.876678 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-md9ft"] Jul 06 18:23:49.878749 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:23:49.878714 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8261c8d6_57aa_4e80_a01a_b8a33e75f1f2.slice/crio-276c8dc8bd8829de61808565f94a8dd9c9af18d38ae2a7762590ba67fc898fbf WatchSource:0}: Error finding container 276c8dc8bd8829de61808565f94a8dd9c9af18d38ae2a7762590ba67fc898fbf: Status 404 returned error can't find the container with id 276c8dc8bd8829de61808565f94a8dd9c9af18d38ae2a7762590ba67fc898fbf Jul 06 18:23:50.054317 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:50.054229 2566 generic.go:356] "Generic (PLEG): container finished" podID="f32c5d84-d81b-4a10-a102-0fd3a939fa75" containerID="9efbdc078cab820526fe04d5ec93a7fdd4a07ce4316bd4d3efd6e59e26be24cd" exitCode=0 Jul 06 18:23:50.054478 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:50.054315 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp" event={"ID":"f32c5d84-d81b-4a10-a102-0fd3a939fa75","Type":"ContainerDied","Data":"9efbdc078cab820526fe04d5ec93a7fdd4a07ce4316bd4d3efd6e59e26be24cd"} Jul 06 18:23:50.054478 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:50.054365 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp" event={"ID":"f32c5d84-d81b-4a10-a102-0fd3a939fa75","Type":"ContainerStarted","Data":"1a7c06e87e524dd1b86962c67828026c8a0b14fc9c8e22a651a4263a5016bfa2"} Jul 06 18:23:50.058171 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:50.058143 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-md9ft" event={"ID":"8261c8d6-57aa-4e80-a01a-b8a33e75f1f2","Type":"ContainerStarted","Data":"276c8dc8bd8829de61808565f94a8dd9c9af18d38ae2a7762590ba67fc898fbf"} Jul 06 18:23:51.120544 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:51.120511 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:23:51.421350 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:51.421297 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-n7r9c"] Jul 06 18:23:51.429094 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:51.429067 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-n7r9c" Jul 06 18:23:51.433593 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:51.433558 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-n7r9c"] Jul 06 18:23:51.434209 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:51.433929 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-nrzfm\"" Jul 06 18:23:51.510389 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:51.510356 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:23:51.517748 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:51.517719 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnv7q\" (UniqueName: \"kubernetes.io/projected/e462199e-7861-48d4-8050-95b71b446635-kube-api-access-vnv7q\") pod \"cert-manager-cainjector-559cdd8c87-n7r9c\" (UID: \"e462199e-7861-48d4-8050-95b71b446635\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n7r9c" Jul 06 18:23:51.517917 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:51.517796 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e462199e-7861-48d4-8050-95b71b446635-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-n7r9c\" (UID: \"e462199e-7861-48d4-8050-95b71b446635\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n7r9c" Jul 06 18:23:51.618643 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:51.618606 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnv7q\" (UniqueName: \"kubernetes.io/projected/e462199e-7861-48d4-8050-95b71b446635-kube-api-access-vnv7q\") pod \"cert-manager-cainjector-559cdd8c87-n7r9c\" (UID: \"e462199e-7861-48d4-8050-95b71b446635\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n7r9c" Jul 06 18:23:51.618823 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:51.618704 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e462199e-7861-48d4-8050-95b71b446635-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-n7r9c\" (UID: \"e462199e-7861-48d4-8050-95b71b446635\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n7r9c" Jul 06 18:23:51.628639 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:51.628609 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e462199e-7861-48d4-8050-95b71b446635-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-n7r9c\" (UID: \"e462199e-7861-48d4-8050-95b71b446635\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n7r9c" Jul 06 18:23:51.628778 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:51.628647 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnv7q\" (UniqueName: \"kubernetes.io/projected/e462199e-7861-48d4-8050-95b71b446635-kube-api-access-vnv7q\") pod \"cert-manager-cainjector-559cdd8c87-n7r9c\" (UID: \"e462199e-7861-48d4-8050-95b71b446635\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n7r9c" Jul 06 18:23:51.743705 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:51.743622 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-n7r9c" Jul 06 18:23:52.826220 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:52.826188 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-n7r9c"] Jul 06 18:23:52.828945 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:23:52.828896 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode462199e_7861_48d4_8050_95b71b446635.slice/crio-7e73f89aabfe9d39648bec96fec7c6c57abcfbc27746ed4b898922ddc9ebda7a WatchSource:0}: Error finding container 7e73f89aabfe9d39648bec96fec7c6c57abcfbc27746ed4b898922ddc9ebda7a: Status 404 returned error can't find the container with id 7e73f89aabfe9d39648bec96fec7c6c57abcfbc27746ed4b898922ddc9ebda7a Jul 06 18:23:53.074925 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:53.074808 2566 generic.go:356] "Generic (PLEG): container finished" podID="f32c5d84-d81b-4a10-a102-0fd3a939fa75" containerID="df3b90d9cf8bb6d4af629ed5b0746bb69a52fb7bc44c939a90a6eca76ee7dd5f" exitCode=0 Jul 06 18:23:53.075111 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:53.074919 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp" event={"ID":"f32c5d84-d81b-4a10-a102-0fd3a939fa75","Type":"ContainerDied","Data":"df3b90d9cf8bb6d4af629ed5b0746bb69a52fb7bc44c939a90a6eca76ee7dd5f"} Jul 06 18:23:53.077426 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:53.076950 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-md9ft" event={"ID":"8261c8d6-57aa-4e80-a01a-b8a33e75f1f2","Type":"ContainerStarted","Data":"12eb3cc7da257186dceadf46b4bcd35707d61ccf2d5e9218eaeb840fcc5f92f0"} Jul 06 18:23:53.077540 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:53.077453 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-md9ft" Jul 06 18:23:53.079548 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:53.079522 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-n7r9c" event={"ID":"e462199e-7861-48d4-8050-95b71b446635","Type":"ContainerStarted","Data":"aa0f928dd9b62158c1fa9ba57584d0480c346711be9ffc2a826763119c56c34a"} Jul 06 18:23:53.079548 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:53.079556 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-n7r9c" event={"ID":"e462199e-7861-48d4-8050-95b71b446635","Type":"ContainerStarted","Data":"7e73f89aabfe9d39648bec96fec7c6c57abcfbc27746ed4b898922ddc9ebda7a"} Jul 06 18:23:53.116504 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:53.116458 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-n7r9c" podStartSLOduration=2.116441909 podStartE2EDuration="2.116441909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:23:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:23:53.115903726 +0000 UTC m=+249.842332684" watchObservedRunningTime="2026-07-06 18:23:53.116441909 +0000 UTC m=+249.842870861" Jul 06 18:23:53.150990 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:53.150941 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-md9ft" podStartSLOduration=1.33149645 podStartE2EDuration="4.150923185s" totalImagesPullingTime="2.819426735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:23:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:23:53.148175711 +0000 UTC m=+249.874604671" watchObservedRunningTime="2026-07-06 18:23:53.150923185 +0000 UTC m=+249.877352142" Jul 06 18:23:53.825744 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:53.825703 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:23:53.886990 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:53.886951 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:23:54.086489 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:54.086382 2566 generic.go:356] "Generic (PLEG): container finished" podID="f32c5d84-d81b-4a10-a102-0fd3a939fa75" containerID="06e7916a348e676eb552fc170d1c8cf885f5f29269678369509b0936491a591f" exitCode=0 Jul 06 18:23:54.086489 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:54.086422 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp" event={"ID":"f32c5d84-d81b-4a10-a102-0fd3a939fa75","Type":"ContainerDied","Data":"06e7916a348e676eb552fc170d1c8cf885f5f29269678369509b0936491a591f"} Jul 06 18:23:55.223411 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:55.223384 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp" Jul 06 18:23:55.356194 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:55.356087 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qk52b\" (UniqueName: \"kubernetes.io/projected/f32c5d84-d81b-4a10-a102-0fd3a939fa75-kube-api-access-qk52b\") pod \"f32c5d84-d81b-4a10-a102-0fd3a939fa75\" (UID: \"f32c5d84-d81b-4a10-a102-0fd3a939fa75\") " Jul 06 18:23:55.356194 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:55.356142 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f32c5d84-d81b-4a10-a102-0fd3a939fa75-util\") pod \"f32c5d84-d81b-4a10-a102-0fd3a939fa75\" (UID: \"f32c5d84-d81b-4a10-a102-0fd3a939fa75\") " Jul 06 18:23:55.356194 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:55.356184 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f32c5d84-d81b-4a10-a102-0fd3a939fa75-bundle\") pod \"f32c5d84-d81b-4a10-a102-0fd3a939fa75\" (UID: \"f32c5d84-d81b-4a10-a102-0fd3a939fa75\") " Jul 06 18:23:55.356659 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:55.356629 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f32c5d84-d81b-4a10-a102-0fd3a939fa75-bundle" (OuterVolumeSpecName: "bundle") pod "f32c5d84-d81b-4a10-a102-0fd3a939fa75" (UID: "f32c5d84-d81b-4a10-a102-0fd3a939fa75"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:23:55.358497 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:55.358447 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f32c5d84-d81b-4a10-a102-0fd3a939fa75-kube-api-access-qk52b" (OuterVolumeSpecName: "kube-api-access-qk52b") pod "f32c5d84-d81b-4a10-a102-0fd3a939fa75" (UID: "f32c5d84-d81b-4a10-a102-0fd3a939fa75"). InnerVolumeSpecName "kube-api-access-qk52b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:23:55.361944 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:55.361919 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f32c5d84-d81b-4a10-a102-0fd3a939fa75-util" (OuterVolumeSpecName: "util") pod "f32c5d84-d81b-4a10-a102-0fd3a939fa75" (UID: "f32c5d84-d81b-4a10-a102-0fd3a939fa75"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:23:55.457365 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:55.457327 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qk52b\" (UniqueName: \"kubernetes.io/projected/f32c5d84-d81b-4a10-a102-0fd3a939fa75-kube-api-access-qk52b\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:23:55.457365 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:55.457361 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f32c5d84-d81b-4a10-a102-0fd3a939fa75-util\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:23:55.457365 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:55.457374 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f32c5d84-d81b-4a10-a102-0fd3a939fa75-bundle\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:23:56.096564 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:56.096536 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp" Jul 06 18:23:56.096564 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:56.096555 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdrcbp" event={"ID":"f32c5d84-d81b-4a10-a102-0fd3a939fa75","Type":"ContainerDied","Data":"1a7c06e87e524dd1b86962c67828026c8a0b14fc9c8e22a651a4263a5016bfa2"} Jul 06 18:23:56.096758 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:56.096594 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1a7c06e87e524dd1b86962c67828026c8a0b14fc9c8e22a651a4263a5016bfa2" Jul 06 18:23:59.629204 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:59.629169 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:23:59.732424 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:59.732394 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:23:59.785334 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:23:59.785297 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:00.092857 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:00.092823 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-md9ft" Jul 06 18:24:00.564858 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:00.564822 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-m8qtt"] Jul 06 18:24:00.565198 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:00.565169 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f32c5d84-d81b-4a10-a102-0fd3a939fa75" containerName="extract" Jul 06 18:24:00.565198 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:00.565190 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f32c5d84-d81b-4a10-a102-0fd3a939fa75" containerName="extract" Jul 06 18:24:00.565198 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:00.565202 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f32c5d84-d81b-4a10-a102-0fd3a939fa75" containerName="pull" Jul 06 18:24:00.565445 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:00.565210 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f32c5d84-d81b-4a10-a102-0fd3a939fa75" containerName="pull" Jul 06 18:24:00.565445 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:00.565227 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f32c5d84-d81b-4a10-a102-0fd3a939fa75" containerName="util" Jul 06 18:24:00.565445 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:00.565234 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f32c5d84-d81b-4a10-a102-0fd3a939fa75" containerName="util" Jul 06 18:24:00.565445 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:00.565338 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f32c5d84-d81b-4a10-a102-0fd3a939fa75" containerName="extract" Jul 06 18:24:00.567452 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:00.567434 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-m8qtt" Jul 06 18:24:00.570391 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:00.570370 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-ntr8w\"" Jul 06 18:24:00.577400 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:00.577378 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-m8qtt"] Jul 06 18:24:00.706025 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:00.705986 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/adbeb7dd-3db7-4020-9466-56457b57ec04-bound-sa-token\") pod \"cert-manager-55b4d954d5-m8qtt\" (UID: \"adbeb7dd-3db7-4020-9466-56457b57ec04\") " pod="cert-manager/cert-manager-55b4d954d5-m8qtt" Jul 06 18:24:00.706025 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:00.706022 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnznl\" (UniqueName: \"kubernetes.io/projected/adbeb7dd-3db7-4020-9466-56457b57ec04-kube-api-access-jnznl\") pod \"cert-manager-55b4d954d5-m8qtt\" (UID: \"adbeb7dd-3db7-4020-9466-56457b57ec04\") " pod="cert-manager/cert-manager-55b4d954d5-m8qtt" Jul 06 18:24:00.806843 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:00.806805 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/adbeb7dd-3db7-4020-9466-56457b57ec04-bound-sa-token\") pod \"cert-manager-55b4d954d5-m8qtt\" (UID: \"adbeb7dd-3db7-4020-9466-56457b57ec04\") " pod="cert-manager/cert-manager-55b4d954d5-m8qtt" Jul 06 18:24:00.806843 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:00.806843 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnznl\" (UniqueName: \"kubernetes.io/projected/adbeb7dd-3db7-4020-9466-56457b57ec04-kube-api-access-jnznl\") pod \"cert-manager-55b4d954d5-m8qtt\" (UID: \"adbeb7dd-3db7-4020-9466-56457b57ec04\") " pod="cert-manager/cert-manager-55b4d954d5-m8qtt" Jul 06 18:24:00.815577 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:00.815508 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnznl\" (UniqueName: \"kubernetes.io/projected/adbeb7dd-3db7-4020-9466-56457b57ec04-kube-api-access-jnznl\") pod \"cert-manager-55b4d954d5-m8qtt\" (UID: \"adbeb7dd-3db7-4020-9466-56457b57ec04\") " pod="cert-manager/cert-manager-55b4d954d5-m8qtt" Jul 06 18:24:00.815679 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:00.815613 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/adbeb7dd-3db7-4020-9466-56457b57ec04-bound-sa-token\") pod \"cert-manager-55b4d954d5-m8qtt\" (UID: \"adbeb7dd-3db7-4020-9466-56457b57ec04\") " pod="cert-manager/cert-manager-55b4d954d5-m8qtt" Jul 06 18:24:00.877108 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:00.877075 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-m8qtt" Jul 06 18:24:01.019560 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:01.019536 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-m8qtt"] Jul 06 18:24:01.021753 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:24:01.021725 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podadbeb7dd_3db7_4020_9466_56457b57ec04.slice/crio-e23c6d2263d111bd6850c16a8d0c531e971697fd0e3b42e07a37f044804cebb9 WatchSource:0}: Error finding container e23c6d2263d111bd6850c16a8d0c531e971697fd0e3b42e07a37f044804cebb9: Status 404 returned error can't find the container with id e23c6d2263d111bd6850c16a8d0c531e971697fd0e3b42e07a37f044804cebb9 Jul 06 18:24:01.121783 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:01.121750 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-m8qtt" event={"ID":"adbeb7dd-3db7-4020-9466-56457b57ec04","Type":"ContainerStarted","Data":"4db21343511922eb7e6dddc135cfddec82e7122a9b57c1d3f07e1fedb5e63ef1"} Jul 06 18:24:01.121964 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:01.121792 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-m8qtt" event={"ID":"adbeb7dd-3db7-4020-9466-56457b57ec04","Type":"ContainerStarted","Data":"e23c6d2263d111bd6850c16a8d0c531e971697fd0e3b42e07a37f044804cebb9"} Jul 06 18:24:01.142742 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:01.142690 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-m8qtt" podStartSLOduration=1.142676032 podStartE2EDuration="1.142676032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:24:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:24:01.14040054 +0000 UTC m=+257.866829515" watchObservedRunningTime="2026-07-06 18:24:01.142676032 +0000 UTC m=+257.869104990" Jul 06 18:24:05.285028 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:05.284951 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:05.381999 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:05.381965 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:07.461158 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:07.461121 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:07.546314 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:07.546281 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv"] Jul 06 18:24:07.548960 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:07.548944 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv" Jul 06 18:24:07.551786 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:07.551761 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 18:24:07.551942 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:07.551763 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 18:24:07.553086 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:07.553072 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rqp65\"" Jul 06 18:24:07.558355 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:07.558333 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv"] Jul 06 18:24:07.564288 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:07.564264 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/528b351c-1568-42d7-9258-1bd063e3d579-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv\" (UID: \"528b351c-1568-42d7-9258-1bd063e3d579\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv" Jul 06 18:24:07.564402 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:07.564293 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/528b351c-1568-42d7-9258-1bd063e3d579-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv\" (UID: \"528b351c-1568-42d7-9258-1bd063e3d579\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv" Jul 06 18:24:07.564402 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:07.564322 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9kjd\" (UniqueName: \"kubernetes.io/projected/528b351c-1568-42d7-9258-1bd063e3d579-kube-api-access-t9kjd\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv\" (UID: \"528b351c-1568-42d7-9258-1bd063e3d579\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv" Jul 06 18:24:07.665143 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:07.665096 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/528b351c-1568-42d7-9258-1bd063e3d579-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv\" (UID: \"528b351c-1568-42d7-9258-1bd063e3d579\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv" Jul 06 18:24:07.665143 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:07.665136 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/528b351c-1568-42d7-9258-1bd063e3d579-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv\" (UID: \"528b351c-1568-42d7-9258-1bd063e3d579\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv" Jul 06 18:24:07.665387 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:07.665180 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9kjd\" (UniqueName: \"kubernetes.io/projected/528b351c-1568-42d7-9258-1bd063e3d579-kube-api-access-t9kjd\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv\" (UID: \"528b351c-1568-42d7-9258-1bd063e3d579\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv" Jul 06 18:24:07.665540 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:07.665504 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/528b351c-1568-42d7-9258-1bd063e3d579-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv\" (UID: \"528b351c-1568-42d7-9258-1bd063e3d579\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv" Jul 06 18:24:07.665588 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:07.665518 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/528b351c-1568-42d7-9258-1bd063e3d579-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv\" (UID: \"528b351c-1568-42d7-9258-1bd063e3d579\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv" Jul 06 18:24:07.674853 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:07.674823 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9kjd\" (UniqueName: \"kubernetes.io/projected/528b351c-1568-42d7-9258-1bd063e3d579-kube-api-access-t9kjd\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv\" (UID: \"528b351c-1568-42d7-9258-1bd063e3d579\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv" Jul 06 18:24:07.859958 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:07.859851 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv" Jul 06 18:24:08.011614 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:08.011580 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv"] Jul 06 18:24:08.014017 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:24:08.013988 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod528b351c_1568_42d7_9258_1bd063e3d579.slice/crio-860f8eb9a1b218f1bb5c319f067261722d93c5e2ede3e4a687c85f1d768ae7e6 WatchSource:0}: Error finding container 860f8eb9a1b218f1bb5c319f067261722d93c5e2ede3e4a687c85f1d768ae7e6: Status 404 returned error can't find the container with id 860f8eb9a1b218f1bb5c319f067261722d93c5e2ede3e4a687c85f1d768ae7e6 Jul 06 18:24:08.154238 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:08.154201 2566 generic.go:356] "Generic (PLEG): container finished" podID="528b351c-1568-42d7-9258-1bd063e3d579" containerID="cd69481304a914b42aa79acbfd815e6dcbc08c954392c43090a739c69b12b90a" exitCode=0 Jul 06 18:24:08.154426 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:08.154290 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv" event={"ID":"528b351c-1568-42d7-9258-1bd063e3d579","Type":"ContainerDied","Data":"cd69481304a914b42aa79acbfd815e6dcbc08c954392c43090a739c69b12b90a"} Jul 06 18:24:08.154426 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:08.154349 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv" event={"ID":"528b351c-1568-42d7-9258-1bd063e3d579","Type":"ContainerStarted","Data":"860f8eb9a1b218f1bb5c319f067261722d93c5e2ede3e4a687c85f1d768ae7e6"} Jul 06 18:24:09.160981 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:09.160943 2566 generic.go:356] "Generic (PLEG): container finished" podID="528b351c-1568-42d7-9258-1bd063e3d579" containerID="beaad7f50fd80fe9dda5e90e87d832dd8c7482d086e60e56314504b8a1099ce3" exitCode=0 Jul 06 18:24:09.161501 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:09.161038 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv" event={"ID":"528b351c-1568-42d7-9258-1bd063e3d579","Type":"ContainerDied","Data":"beaad7f50fd80fe9dda5e90e87d832dd8c7482d086e60e56314504b8a1099ce3"} Jul 06 18:24:10.168013 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:10.167979 2566 generic.go:356] "Generic (PLEG): container finished" podID="528b351c-1568-42d7-9258-1bd063e3d579" containerID="fd3a2534f8f2ebda80ea29993df525a63a28798244e2650b72c8889bf919ca5a" exitCode=0 Jul 06 18:24:10.168392 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:10.168033 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv" event={"ID":"528b351c-1568-42d7-9258-1bd063e3d579","Type":"ContainerDied","Data":"fd3a2534f8f2ebda80ea29993df525a63a28798244e2650b72c8889bf919ca5a"} Jul 06 18:24:11.300915 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:11.300890 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv" Jul 06 18:24:11.398147 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:11.398107 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9kjd\" (UniqueName: \"kubernetes.io/projected/528b351c-1568-42d7-9258-1bd063e3d579-kube-api-access-t9kjd\") pod \"528b351c-1568-42d7-9258-1bd063e3d579\" (UID: \"528b351c-1568-42d7-9258-1bd063e3d579\") " Jul 06 18:24:11.398315 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:11.398175 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/528b351c-1568-42d7-9258-1bd063e3d579-bundle\") pod \"528b351c-1568-42d7-9258-1bd063e3d579\" (UID: \"528b351c-1568-42d7-9258-1bd063e3d579\") " Jul 06 18:24:11.398315 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:11.398230 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/528b351c-1568-42d7-9258-1bd063e3d579-util\") pod \"528b351c-1568-42d7-9258-1bd063e3d579\" (UID: \"528b351c-1568-42d7-9258-1bd063e3d579\") " Jul 06 18:24:11.398969 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:11.398937 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/528b351c-1568-42d7-9258-1bd063e3d579-bundle" (OuterVolumeSpecName: "bundle") pod "528b351c-1568-42d7-9258-1bd063e3d579" (UID: "528b351c-1568-42d7-9258-1bd063e3d579"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:24:11.400371 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:11.400336 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/528b351c-1568-42d7-9258-1bd063e3d579-kube-api-access-t9kjd" (OuterVolumeSpecName: "kube-api-access-t9kjd") pod "528b351c-1568-42d7-9258-1bd063e3d579" (UID: "528b351c-1568-42d7-9258-1bd063e3d579"). InnerVolumeSpecName "kube-api-access-t9kjd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:24:11.404280 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:11.404259 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/528b351c-1568-42d7-9258-1bd063e3d579-util" (OuterVolumeSpecName: "util") pod "528b351c-1568-42d7-9258-1bd063e3d579" (UID: "528b351c-1568-42d7-9258-1bd063e3d579"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:24:11.499443 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:11.499349 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9kjd\" (UniqueName: \"kubernetes.io/projected/528b351c-1568-42d7-9258-1bd063e3d579-kube-api-access-t9kjd\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:24:11.499443 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:11.499383 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/528b351c-1568-42d7-9258-1bd063e3d579-bundle\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:24:11.499443 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:11.499391 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/528b351c-1568-42d7-9258-1bd063e3d579-util\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:24:12.178790 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:12.178756 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv" event={"ID":"528b351c-1568-42d7-9258-1bd063e3d579","Type":"ContainerDied","Data":"860f8eb9a1b218f1bb5c319f067261722d93c5e2ede3e4a687c85f1d768ae7e6"} Jul 06 18:24:12.178990 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:12.178803 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="860f8eb9a1b218f1bb5c319f067261722d93c5e2ede3e4a687c85f1d768ae7e6" Jul 06 18:24:12.178990 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:12.178773 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnzdvv" Jul 06 18:24:20.247471 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:20.247438 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:20.353046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:20.353011 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:20.841083 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:20.841046 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:21.064328 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.064292 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:21.166928 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.166862 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:21.218633 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.218598 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bf95b557-trqql"] Jul 06 18:24:21.218957 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.218944 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="528b351c-1568-42d7-9258-1bd063e3d579" containerName="extract" Jul 06 18:24:21.219004 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.218959 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="528b351c-1568-42d7-9258-1bd063e3d579" containerName="extract" Jul 06 18:24:21.219004 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.218968 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="528b351c-1568-42d7-9258-1bd063e3d579" containerName="pull" Jul 06 18:24:21.219004 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.218974 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="528b351c-1568-42d7-9258-1bd063e3d579" containerName="pull" Jul 06 18:24:21.219004 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.218983 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="528b351c-1568-42d7-9258-1bd063e3d579" containerName="util" Jul 06 18:24:21.219004 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.218989 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="528b351c-1568-42d7-9258-1bd063e3d579" containerName="util" Jul 06 18:24:21.219148 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.219046 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="528b351c-1568-42d7-9258-1bd063e3d579" containerName="extract" Jul 06 18:24:21.224185 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.224163 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-trqql" Jul 06 18:24:21.227664 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.227641 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 18:24:21.227799 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.227673 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 18:24:21.227799 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.227725 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 18:24:21.227924 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.227807 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 18:24:21.228274 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.228256 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mm5zn\"" Jul 06 18:24:21.235939 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.235917 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bf95b557-trqql"] Jul 06 18:24:21.289340 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.289300 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b7b6e07-7bbd-4fe8-985e-d11852b486e8-tmp\") pod \"opendatahub-operator-controller-manager-bf95b557-trqql\" (UID: \"0b7b6e07-7bbd-4fe8-985e-d11852b486e8\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-trqql" Jul 06 18:24:21.289340 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.289341 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmtrg\" (UniqueName: \"kubernetes.io/projected/0b7b6e07-7bbd-4fe8-985e-d11852b486e8-kube-api-access-vmtrg\") pod \"opendatahub-operator-controller-manager-bf95b557-trqql\" (UID: \"0b7b6e07-7bbd-4fe8-985e-d11852b486e8\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-trqql" Jul 06 18:24:21.289753 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.289368 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b7b6e07-7bbd-4fe8-985e-d11852b486e8-webhook-cert\") pod \"opendatahub-operator-controller-manager-bf95b557-trqql\" (UID: \"0b7b6e07-7bbd-4fe8-985e-d11852b486e8\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-trqql" Jul 06 18:24:21.289753 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.289448 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0b7b6e07-7bbd-4fe8-985e-d11852b486e8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bf95b557-trqql\" (UID: \"0b7b6e07-7bbd-4fe8-985e-d11852b486e8\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-trqql" Jul 06 18:24:21.289753 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.289489 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0b7b6e07-7bbd-4fe8-985e-d11852b486e8-manifests\") pod \"opendatahub-operator-controller-manager-bf95b557-trqql\" (UID: \"0b7b6e07-7bbd-4fe8-985e-d11852b486e8\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-trqql" Jul 06 18:24:21.360554 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.360519 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:21.390372 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.390343 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b7b6e07-7bbd-4fe8-985e-d11852b486e8-tmp\") pod \"opendatahub-operator-controller-manager-bf95b557-trqql\" (UID: \"0b7b6e07-7bbd-4fe8-985e-d11852b486e8\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-trqql" Jul 06 18:24:21.390475 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.390379 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmtrg\" (UniqueName: \"kubernetes.io/projected/0b7b6e07-7bbd-4fe8-985e-d11852b486e8-kube-api-access-vmtrg\") pod \"opendatahub-operator-controller-manager-bf95b557-trqql\" (UID: \"0b7b6e07-7bbd-4fe8-985e-d11852b486e8\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-trqql" Jul 06 18:24:21.390475 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.390412 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b7b6e07-7bbd-4fe8-985e-d11852b486e8-webhook-cert\") pod \"opendatahub-operator-controller-manager-bf95b557-trqql\" (UID: \"0b7b6e07-7bbd-4fe8-985e-d11852b486e8\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-trqql" Jul 06 18:24:21.390475 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.390450 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0b7b6e07-7bbd-4fe8-985e-d11852b486e8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bf95b557-trqql\" (UID: \"0b7b6e07-7bbd-4fe8-985e-d11852b486e8\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-trqql" Jul 06 18:24:21.390624 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.390489 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0b7b6e07-7bbd-4fe8-985e-d11852b486e8-manifests\") pod \"opendatahub-operator-controller-manager-bf95b557-trqql\" (UID: \"0b7b6e07-7bbd-4fe8-985e-d11852b486e8\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-trqql" Jul 06 18:24:21.390923 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.390901 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0b7b6e07-7bbd-4fe8-985e-d11852b486e8-manifests\") pod \"opendatahub-operator-controller-manager-bf95b557-trqql\" (UID: \"0b7b6e07-7bbd-4fe8-985e-d11852b486e8\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-trqql" Jul 06 18:24:21.392837 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.392813 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b7b6e07-7bbd-4fe8-985e-d11852b486e8-tmp\") pod \"opendatahub-operator-controller-manager-bf95b557-trqql\" (UID: \"0b7b6e07-7bbd-4fe8-985e-d11852b486e8\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-trqql" Jul 06 18:24:21.393061 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.393041 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0b7b6e07-7bbd-4fe8-985e-d11852b486e8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bf95b557-trqql\" (UID: \"0b7b6e07-7bbd-4fe8-985e-d11852b486e8\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-trqql" Jul 06 18:24:21.393108 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.393076 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b7b6e07-7bbd-4fe8-985e-d11852b486e8-webhook-cert\") pod \"opendatahub-operator-controller-manager-bf95b557-trqql\" (UID: \"0b7b6e07-7bbd-4fe8-985e-d11852b486e8\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-trqql" Jul 06 18:24:21.407004 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.406975 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmtrg\" (UniqueName: \"kubernetes.io/projected/0b7b6e07-7bbd-4fe8-985e-d11852b486e8-kube-api-access-vmtrg\") pod \"opendatahub-operator-controller-manager-bf95b557-trqql\" (UID: \"0b7b6e07-7bbd-4fe8-985e-d11852b486e8\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-trqql" Jul 06 18:24:21.465082 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.464996 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:21.536170 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.536136 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-trqql" Jul 06 18:24:21.574450 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.574411 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:21.679227 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.679200 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bf95b557-trqql"] Jul 06 18:24:21.680638 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:24:21.680609 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b7b6e07_7bbd_4fe8_985e_d11852b486e8.slice/crio-5102de4528b9355a2381e5d26ebc99c5e29a6905b3b880fcbda0a9678d9c18e7 WatchSource:0}: Error finding container 5102de4528b9355a2381e5d26ebc99c5e29a6905b3b880fcbda0a9678d9c18e7: Status 404 returned error can't find the container with id 5102de4528b9355a2381e5d26ebc99c5e29a6905b3b880fcbda0a9678d9c18e7 Jul 06 18:24:21.685029 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.685004 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:21.779544 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:21.779457 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:22.231086 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:22.231048 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-trqql" event={"ID":"0b7b6e07-7bbd-4fe8-985e-d11852b486e8","Type":"ContainerStarted","Data":"5102de4528b9355a2381e5d26ebc99c5e29a6905b3b880fcbda0a9678d9c18e7"} Jul 06 18:24:26.253730 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:26.253687 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-trqql" event={"ID":"0b7b6e07-7bbd-4fe8-985e-d11852b486e8","Type":"ContainerStarted","Data":"acc2da4534b84dbef6578f14df0263ef85978e20c946874a94ab8f615d964fde"} Jul 06 18:24:27.259572 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:27.259533 2566 generic.go:356] "Generic (PLEG): container finished" podID="0b7b6e07-7bbd-4fe8-985e-d11852b486e8" containerID="acc2da4534b84dbef6578f14df0263ef85978e20c946874a94ab8f615d964fde" exitCode=0 Jul 06 18:24:27.259982 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:27.259624 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-trqql" event={"ID":"0b7b6e07-7bbd-4fe8-985e-d11852b486e8","Type":"ContainerDied","Data":"acc2da4534b84dbef6578f14df0263ef85978e20c946874a94ab8f615d964fde"} Jul 06 18:24:28.266734 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:28.266694 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-trqql" event={"ID":"0b7b6e07-7bbd-4fe8-985e-d11852b486e8","Type":"ContainerStarted","Data":"ddd35f8c409c34b0b80c3c19350a7d231c316f6eb828436333b52a0e0363e72b"} Jul 06 18:24:28.267135 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:28.266781 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-trqql" Jul 06 18:24:28.290505 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:28.290451 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-trqql" podStartSLOduration=1.60881634 podStartE2EDuration="7.290431808s" totalImagesPullingTime="4.673915681s" totalInitContainerRuntime="1.007699787s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:24:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:24:28.288988174 +0000 UTC m=+285.015417132" watchObservedRunningTime="2026-07-06 18:24:28.290431808 +0000 UTC m=+285.016860767" Jul 06 18:24:29.697705 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:29.697648 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:31.218218 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:31.218149 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:39.273696 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:39.273664 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-trqql" Jul 06 18:24:40.249520 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:40.249485 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-77945c4cd8-7lw77"] Jul 06 18:24:40.252693 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:40.252674 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-77945c4cd8-7lw77" Jul 06 18:24:40.255699 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:40.255676 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 18:24:40.257167 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:40.257138 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 06 18:24:40.257167 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:40.257138 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-wkwhz\"" Jul 06 18:24:40.257356 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:40.257170 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 06 18:24:40.257356 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:40.257145 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 18:24:40.257356 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:40.257190 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 06 18:24:40.261336 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:40.261310 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-77945c4cd8-7lw77"] Jul 06 18:24:40.271798 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:40.271767 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d1284b93-b89f-4487-b1c4-6cf7e61045f1-metrics-cert\") pod \"lws-controller-manager-77945c4cd8-7lw77\" (UID: \"d1284b93-b89f-4487-b1c4-6cf7e61045f1\") " pod="openshift-lws-operator/lws-controller-manager-77945c4cd8-7lw77" Jul 06 18:24:40.271798 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:40.271803 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d1284b93-b89f-4487-b1c4-6cf7e61045f1-manager-config\") pod \"lws-controller-manager-77945c4cd8-7lw77\" (UID: \"d1284b93-b89f-4487-b1c4-6cf7e61045f1\") " pod="openshift-lws-operator/lws-controller-manager-77945c4cd8-7lw77" Jul 06 18:24:40.272068 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:40.271824 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1284b93-b89f-4487-b1c4-6cf7e61045f1-cert\") pod \"lws-controller-manager-77945c4cd8-7lw77\" (UID: \"d1284b93-b89f-4487-b1c4-6cf7e61045f1\") " pod="openshift-lws-operator/lws-controller-manager-77945c4cd8-7lw77" Jul 06 18:24:40.272068 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:40.271842 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk9lw\" (UniqueName: \"kubernetes.io/projected/d1284b93-b89f-4487-b1c4-6cf7e61045f1-kube-api-access-sk9lw\") pod \"lws-controller-manager-77945c4cd8-7lw77\" (UID: \"d1284b93-b89f-4487-b1c4-6cf7e61045f1\") " pod="openshift-lws-operator/lws-controller-manager-77945c4cd8-7lw77" Jul 06 18:24:40.372516 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:40.372470 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d1284b93-b89f-4487-b1c4-6cf7e61045f1-metrics-cert\") pod \"lws-controller-manager-77945c4cd8-7lw77\" (UID: \"d1284b93-b89f-4487-b1c4-6cf7e61045f1\") " pod="openshift-lws-operator/lws-controller-manager-77945c4cd8-7lw77" Jul 06 18:24:40.372516 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:40.372519 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d1284b93-b89f-4487-b1c4-6cf7e61045f1-manager-config\") pod \"lws-controller-manager-77945c4cd8-7lw77\" (UID: \"d1284b93-b89f-4487-b1c4-6cf7e61045f1\") " pod="openshift-lws-operator/lws-controller-manager-77945c4cd8-7lw77" Jul 06 18:24:40.372998 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:40.372556 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1284b93-b89f-4487-b1c4-6cf7e61045f1-cert\") pod \"lws-controller-manager-77945c4cd8-7lw77\" (UID: \"d1284b93-b89f-4487-b1c4-6cf7e61045f1\") " pod="openshift-lws-operator/lws-controller-manager-77945c4cd8-7lw77" Jul 06 18:24:40.372998 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:40.372716 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk9lw\" (UniqueName: \"kubernetes.io/projected/d1284b93-b89f-4487-b1c4-6cf7e61045f1-kube-api-access-sk9lw\") pod \"lws-controller-manager-77945c4cd8-7lw77\" (UID: \"d1284b93-b89f-4487-b1c4-6cf7e61045f1\") " pod="openshift-lws-operator/lws-controller-manager-77945c4cd8-7lw77" Jul 06 18:24:40.373276 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:40.373257 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d1284b93-b89f-4487-b1c4-6cf7e61045f1-manager-config\") pod \"lws-controller-manager-77945c4cd8-7lw77\" (UID: \"d1284b93-b89f-4487-b1c4-6cf7e61045f1\") " pod="openshift-lws-operator/lws-controller-manager-77945c4cd8-7lw77" Jul 06 18:24:40.375396 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:40.375370 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d1284b93-b89f-4487-b1c4-6cf7e61045f1-metrics-cert\") pod \"lws-controller-manager-77945c4cd8-7lw77\" (UID: \"d1284b93-b89f-4487-b1c4-6cf7e61045f1\") " pod="openshift-lws-operator/lws-controller-manager-77945c4cd8-7lw77" Jul 06 18:24:40.375511 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:40.375373 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1284b93-b89f-4487-b1c4-6cf7e61045f1-cert\") pod \"lws-controller-manager-77945c4cd8-7lw77\" (UID: \"d1284b93-b89f-4487-b1c4-6cf7e61045f1\") " pod="openshift-lws-operator/lws-controller-manager-77945c4cd8-7lw77" Jul 06 18:24:40.381262 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:40.381237 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk9lw\" (UniqueName: \"kubernetes.io/projected/d1284b93-b89f-4487-b1c4-6cf7e61045f1-kube-api-access-sk9lw\") pod \"lws-controller-manager-77945c4cd8-7lw77\" (UID: \"d1284b93-b89f-4487-b1c4-6cf7e61045f1\") " pod="openshift-lws-operator/lws-controller-manager-77945c4cd8-7lw77" Jul 06 18:24:40.563472 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:40.563378 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-77945c4cd8-7lw77" Jul 06 18:24:40.706079 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:40.706051 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-77945c4cd8-7lw77"] Jul 06 18:24:40.707455 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:24:40.707424 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1284b93_b89f_4487_b1c4_6cf7e61045f1.slice/crio-b2fb5efa7041c8b2a13dc92289ad13f57e21a1b7cb1f677395161abb6aa8b6ee WatchSource:0}: Error finding container b2fb5efa7041c8b2a13dc92289ad13f57e21a1b7cb1f677395161abb6aa8b6ee: Status 404 returned error can't find the container with id b2fb5efa7041c8b2a13dc92289ad13f57e21a1b7cb1f677395161abb6aa8b6ee Jul 06 18:24:40.956449 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:40.956418 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:41.151892 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:41.151822 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:41.332076 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:41.331989 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-77945c4cd8-7lw77" event={"ID":"d1284b93-b89f-4487-b1c4-6cf7e61045f1","Type":"ContainerStarted","Data":"b2fb5efa7041c8b2a13dc92289ad13f57e21a1b7cb1f677395161abb6aa8b6ee"} Jul 06 18:24:41.761388 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:41.761308 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:41.862996 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:41.862964 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:41.965955 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:41.965920 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:42.178409 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:42.178372 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:42.338251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:42.338214 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-77945c4cd8-7lw77" event={"ID":"d1284b93-b89f-4487-b1c4-6cf7e61045f1","Type":"ContainerStarted","Data":"2db72a5b44ef91329993dcef9cc197aac29a0c29834495b94191c8456e9b4e19"} Jul 06 18:24:42.338436 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:42.338264 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-77945c4cd8-7lw77" Jul 06 18:24:42.352562 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:42.352497 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:42.370182 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:42.370125 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-77945c4cd8-7lw77" podStartSLOduration=1.240174218 podStartE2EDuration="2.370110816s" totalImagesPullingTime="1.129936598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:24:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:24:42.367461793 +0000 UTC m=+299.093890752" watchObservedRunningTime="2026-07-06 18:24:42.370110816 +0000 UTC m=+299.096539774" Jul 06 18:24:42.522788 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:42.522680 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:43.694355 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:43.694330 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffzjr_1ede441d-79f2-4365-8a08-403bf043ef73/ovn-acl-logging/0.log" Jul 06 18:24:43.698096 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:43.698073 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffzjr_1ede441d-79f2-4365-8a08-403bf043ef73/ovn-acl-logging/0.log" Jul 06 18:24:43.699170 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:43.699151 2566 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 18:24:51.978210 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:51.978170 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:52.082992 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:52.082963 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:52.187691 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:52.187660 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:52.461668 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:52.461631 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:52.561571 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:52.561539 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:53.345289 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:53.345252 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-77945c4cd8-7lw77" Jul 06 18:24:53.403978 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:53.403936 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:53.617651 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:53.617554 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:53.765235 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:53.765190 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:54.136913 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:54.136857 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:54.285772 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:54.285742 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:55.465603 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:55.465567 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:59.458036 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:59.458004 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:24:59.558188 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:24:59.558155 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:25:00.219368 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.219308 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr"] Jul 06 18:25:00.226567 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.226540 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:00.229504 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.229479 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 18:25:00.229677 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.229518 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 18:25:00.229677 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.229566 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 18:25:00.229801 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.229748 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-4f8lk\"" Jul 06 18:25:00.233613 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.233589 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr"] Jul 06 18:25:00.249206 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.249176 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:25:00.342986 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.342958 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e40633c-06a3-47fa-8c37-1401dc956580-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jvrqr\" (UID: \"6e40633c-06a3-47fa-8c37-1401dc956580\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:00.343177 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.343002 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v89xv\" (UniqueName: \"kubernetes.io/projected/6e40633c-06a3-47fa-8c37-1401dc956580-kube-api-access-v89xv\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jvrqr\" (UID: \"6e40633c-06a3-47fa-8c37-1401dc956580\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:00.343177 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.343036 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e40633c-06a3-47fa-8c37-1401dc956580-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jvrqr\" (UID: \"6e40633c-06a3-47fa-8c37-1401dc956580\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:00.343177 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.343064 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e40633c-06a3-47fa-8c37-1401dc956580-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jvrqr\" (UID: \"6e40633c-06a3-47fa-8c37-1401dc956580\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:00.343177 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.343083 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e40633c-06a3-47fa-8c37-1401dc956580-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jvrqr\" (UID: \"6e40633c-06a3-47fa-8c37-1401dc956580\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:00.343177 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.343147 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e40633c-06a3-47fa-8c37-1401dc956580-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jvrqr\" (UID: \"6e40633c-06a3-47fa-8c37-1401dc956580\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:00.343423 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.343262 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e40633c-06a3-47fa-8c37-1401dc956580-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jvrqr\" (UID: \"6e40633c-06a3-47fa-8c37-1401dc956580\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:00.343423 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.343286 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e40633c-06a3-47fa-8c37-1401dc956580-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jvrqr\" (UID: \"6e40633c-06a3-47fa-8c37-1401dc956580\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:00.343423 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.343330 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e40633c-06a3-47fa-8c37-1401dc956580-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jvrqr\" (UID: \"6e40633c-06a3-47fa-8c37-1401dc956580\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:00.348980 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.348958 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:25:00.444340 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.444309 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e40633c-06a3-47fa-8c37-1401dc956580-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jvrqr\" (UID: \"6e40633c-06a3-47fa-8c37-1401dc956580\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:00.444521 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.444345 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e40633c-06a3-47fa-8c37-1401dc956580-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jvrqr\" (UID: \"6e40633c-06a3-47fa-8c37-1401dc956580\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:00.444521 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.444378 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e40633c-06a3-47fa-8c37-1401dc956580-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jvrqr\" (UID: \"6e40633c-06a3-47fa-8c37-1401dc956580\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:00.444521 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.444492 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e40633c-06a3-47fa-8c37-1401dc956580-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jvrqr\" (UID: \"6e40633c-06a3-47fa-8c37-1401dc956580\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:00.444684 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.444525 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e40633c-06a3-47fa-8c37-1401dc956580-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jvrqr\" (UID: \"6e40633c-06a3-47fa-8c37-1401dc956580\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:00.444684 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.444571 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e40633c-06a3-47fa-8c37-1401dc956580-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jvrqr\" (UID: \"6e40633c-06a3-47fa-8c37-1401dc956580\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:00.444684 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.444602 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e40633c-06a3-47fa-8c37-1401dc956580-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jvrqr\" (UID: \"6e40633c-06a3-47fa-8c37-1401dc956580\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:00.444684 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.444654 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v89xv\" (UniqueName: \"kubernetes.io/projected/6e40633c-06a3-47fa-8c37-1401dc956580-kube-api-access-v89xv\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jvrqr\" (UID: \"6e40633c-06a3-47fa-8c37-1401dc956580\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:00.444893 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.444713 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e40633c-06a3-47fa-8c37-1401dc956580-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jvrqr\" (UID: \"6e40633c-06a3-47fa-8c37-1401dc956580\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:00.444893 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.444836 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e40633c-06a3-47fa-8c37-1401dc956580-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jvrqr\" (UID: \"6e40633c-06a3-47fa-8c37-1401dc956580\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:00.445018 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.444931 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e40633c-06a3-47fa-8c37-1401dc956580-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jvrqr\" (UID: \"6e40633c-06a3-47fa-8c37-1401dc956580\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:00.445018 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.444962 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e40633c-06a3-47fa-8c37-1401dc956580-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jvrqr\" (UID: \"6e40633c-06a3-47fa-8c37-1401dc956580\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:00.445191 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.445168 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e40633c-06a3-47fa-8c37-1401dc956580-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jvrqr\" (UID: \"6e40633c-06a3-47fa-8c37-1401dc956580\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:00.445421 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.445400 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e40633c-06a3-47fa-8c37-1401dc956580-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jvrqr\" (UID: \"6e40633c-06a3-47fa-8c37-1401dc956580\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:00.446262 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.446236 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:25:00.446862 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.446845 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e40633c-06a3-47fa-8c37-1401dc956580-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jvrqr\" (UID: \"6e40633c-06a3-47fa-8c37-1401dc956580\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:00.447293 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.447273 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e40633c-06a3-47fa-8c37-1401dc956580-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jvrqr\" (UID: \"6e40633c-06a3-47fa-8c37-1401dc956580\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:00.453220 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.453197 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e40633c-06a3-47fa-8c37-1401dc956580-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jvrqr\" (UID: \"6e40633c-06a3-47fa-8c37-1401dc956580\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:00.454120 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.454100 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v89xv\" (UniqueName: \"kubernetes.io/projected/6e40633c-06a3-47fa-8c37-1401dc956580-kube-api-access-v89xv\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jvrqr\" (UID: \"6e40633c-06a3-47fa-8c37-1401dc956580\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:00.538097 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.538009 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:00.725266 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.725238 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr"] Jul 06 18:25:00.727155 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:25:00.727122 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e40633c_06a3_47fa_8c37_1401dc956580.slice/crio-e059e0597a86426a690de41f59d3b558ad565c8882103cb6a18ca9e9b428190c WatchSource:0}: Error finding container e059e0597a86426a690de41f59d3b558ad565c8882103cb6a18ca9e9b428190c: Status 404 returned error can't find the container with id e059e0597a86426a690de41f59d3b558ad565c8882103cb6a18ca9e9b428190c Jul 06 18:25:00.729413 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:00.729393 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 18:25:01.428430 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:01.428390 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" event={"ID":"6e40633c-06a3-47fa-8c37-1401dc956580","Type":"ContainerStarted","Data":"e059e0597a86426a690de41f59d3b558ad565c8882103cb6a18ca9e9b428190c"} Jul 06 18:25:05.206297 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:05.206256 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:25:05.206588 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:05.206331 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:25:05.206588 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:05.206360 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:25:05.447801 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:05.447766 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" event={"ID":"6e40633c-06a3-47fa-8c37-1401dc956580","Type":"ContainerStarted","Data":"4e1eeb0430342dc42e2595d5115ea59566f2b22bd3fe083fc559203d0cf8a6d3"} Jul 06 18:25:05.468823 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:05.468706 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" podStartSLOduration=0.992242097 podStartE2EDuration="5.468685217s" totalImagesPullingTime="4.47644312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:25:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:25:05.46758619 +0000 UTC m=+322.194015163" watchObservedRunningTime="2026-07-06 18:25:05.468685217 +0000 UTC m=+322.195114178" Jul 06 18:25:05.539066 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:05.539029 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:06.543235 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:06.543203 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:07.458113 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:07.458084 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:07.458993 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:07.458976 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jvrqr" Jul 06 18:25:42.639799 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:42.639708 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:25:42.733106 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:42.733075 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:25:42.766804 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:42.766772 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7vb58"] Jul 06 18:25:42.772841 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:42.772818 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7vb58" Jul 06 18:25:42.776852 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:42.776823 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 18:25:42.777343 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:42.777320 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 18:25:42.778196 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:42.778177 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-nglvx\"" Jul 06 18:25:42.780327 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:42.780300 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7vb58"] Jul 06 18:25:42.830057 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:42.830031 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:25:42.834217 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:42.834189 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv2j8\" (UniqueName: \"kubernetes.io/projected/e5ef0634-9222-4dd1-be89-2d440ea6be68-kube-api-access-sv2j8\") pod \"kuadrant-operator-catalog-7vb58\" (UID: \"e5ef0634-9222-4dd1-be89-2d440ea6be68\") " pod="kuadrant-system/kuadrant-operator-catalog-7vb58" Jul 06 18:25:42.935740 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:42.935657 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv2j8\" (UniqueName: \"kubernetes.io/projected/e5ef0634-9222-4dd1-be89-2d440ea6be68-kube-api-access-sv2j8\") pod \"kuadrant-operator-catalog-7vb58\" (UID: \"e5ef0634-9222-4dd1-be89-2d440ea6be68\") " pod="kuadrant-system/kuadrant-operator-catalog-7vb58" Jul 06 18:25:42.949749 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:42.949720 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv2j8\" (UniqueName: \"kubernetes.io/projected/e5ef0634-9222-4dd1-be89-2d440ea6be68-kube-api-access-sv2j8\") pod \"kuadrant-operator-catalog-7vb58\" (UID: \"e5ef0634-9222-4dd1-be89-2d440ea6be68\") " pod="kuadrant-system/kuadrant-operator-catalog-7vb58" Jul 06 18:25:43.085246 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:43.085215 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7vb58" Jul 06 18:25:43.129440 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:43.129402 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7vb58"] Jul 06 18:25:43.218145 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:43.218116 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7vb58"] Jul 06 18:25:43.220066 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:25:43.220039 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5ef0634_9222_4dd1_be89_2d440ea6be68.slice/crio-2e7dc910153fb1ec5f1e412066c7cb4aab89bb8e66e18b86927908d386bd6155 WatchSource:0}: Error finding container 2e7dc910153fb1ec5f1e412066c7cb4aab89bb8e66e18b86927908d386bd6155: Status 404 returned error can't find the container with id 2e7dc910153fb1ec5f1e412066c7cb4aab89bb8e66e18b86927908d386bd6155 Jul 06 18:25:43.338712 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:43.338666 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4dc6g"] Jul 06 18:25:43.342207 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:43.342187 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4dc6g" Jul 06 18:25:43.349323 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:43.349292 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4dc6g"] Jul 06 18:25:43.439635 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:43.439596 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m26b7\" (UniqueName: \"kubernetes.io/projected/e383c0fd-d33c-45bb-a05c-dbce41756fbf-kube-api-access-m26b7\") pod \"kuadrant-operator-catalog-4dc6g\" (UID: \"e383c0fd-d33c-45bb-a05c-dbce41756fbf\") " pod="kuadrant-system/kuadrant-operator-catalog-4dc6g" Jul 06 18:25:43.541028 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:43.540948 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m26b7\" (UniqueName: \"kubernetes.io/projected/e383c0fd-d33c-45bb-a05c-dbce41756fbf-kube-api-access-m26b7\") pod \"kuadrant-operator-catalog-4dc6g\" (UID: \"e383c0fd-d33c-45bb-a05c-dbce41756fbf\") " pod="kuadrant-system/kuadrant-operator-catalog-4dc6g" Jul 06 18:25:43.553576 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:43.553546 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m26b7\" (UniqueName: \"kubernetes.io/projected/e383c0fd-d33c-45bb-a05c-dbce41756fbf-kube-api-access-m26b7\") pod \"kuadrant-operator-catalog-4dc6g\" (UID: \"e383c0fd-d33c-45bb-a05c-dbce41756fbf\") " pod="kuadrant-system/kuadrant-operator-catalog-4dc6g" Jul 06 18:25:43.635440 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:43.635403 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7vb58" event={"ID":"e5ef0634-9222-4dd1-be89-2d440ea6be68","Type":"ContainerStarted","Data":"2e7dc910153fb1ec5f1e412066c7cb4aab89bb8e66e18b86927908d386bd6155"} Jul 06 18:25:43.653047 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:43.653019 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4dc6g" Jul 06 18:25:43.796697 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:43.796668 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4dc6g"] Jul 06 18:25:43.798095 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:25:43.798069 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode383c0fd_d33c_45bb_a05c_dbce41756fbf.slice/crio-aae45ee41c01c0c536ad8e4d8e3118ec9f96090cc27f58ab4a2c034a0cd62848 WatchSource:0}: Error finding container aae45ee41c01c0c536ad8e4d8e3118ec9f96090cc27f58ab4a2c034a0cd62848: Status 404 returned error can't find the container with id aae45ee41c01c0c536ad8e4d8e3118ec9f96090cc27f58ab4a2c034a0cd62848 Jul 06 18:25:44.642822 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:44.642786 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4dc6g" event={"ID":"e383c0fd-d33c-45bb-a05c-dbce41756fbf","Type":"ContainerStarted","Data":"aae45ee41c01c0c536ad8e4d8e3118ec9f96090cc27f58ab4a2c034a0cd62848"} Jul 06 18:25:45.650240 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:45.650199 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4dc6g" event={"ID":"e383c0fd-d33c-45bb-a05c-dbce41756fbf","Type":"ContainerStarted","Data":"042d3833d179fe5848ecfa267ee7ec52f233a81d21d1b25d6e184ce4948cf25f"} Jul 06 18:25:45.651814 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:45.651789 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7vb58" event={"ID":"e5ef0634-9222-4dd1-be89-2d440ea6be68","Type":"ContainerStarted","Data":"6f5f7dc76a39029f415964e6315c0ede29a5425736a7536469fcbb2a033865f8"} Jul 06 18:25:45.651954 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:45.651830 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-7vb58" podUID="e5ef0634-9222-4dd1-be89-2d440ea6be68" containerName="registry-server" containerID="cri-o://6f5f7dc76a39029f415964e6315c0ede29a5425736a7536469fcbb2a033865f8" gracePeriod=2 Jul 06 18:25:45.667737 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:45.667686 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-4dc6g" podStartSLOduration=0.956676703 podStartE2EDuration="2.667672178s" totalImagesPullingTime="1.710995475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:25:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:25:45.665512754 +0000 UTC m=+362.391941714" watchObservedRunningTime="2026-07-06 18:25:45.667672178 +0000 UTC m=+362.394101136" Jul 06 18:25:45.682454 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:45.682406 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-7vb58" podStartSLOduration=1.43236145 podStartE2EDuration="3.682389098s" totalImagesPullingTime="2.250027648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:25:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:25:45.680647549 +0000 UTC m=+362.407076512" watchObservedRunningTime="2026-07-06 18:25:45.682389098 +0000 UTC m=+362.408818055" Jul 06 18:25:45.902199 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:45.902171 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7vb58" Jul 06 18:25:45.964621 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:45.964580 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sv2j8\" (UniqueName: \"kubernetes.io/projected/e5ef0634-9222-4dd1-be89-2d440ea6be68-kube-api-access-sv2j8\") pod \"e5ef0634-9222-4dd1-be89-2d440ea6be68\" (UID: \"e5ef0634-9222-4dd1-be89-2d440ea6be68\") " Jul 06 18:25:45.966883 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:45.966842 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5ef0634-9222-4dd1-be89-2d440ea6be68-kube-api-access-sv2j8" (OuterVolumeSpecName: "kube-api-access-sv2j8") pod "e5ef0634-9222-4dd1-be89-2d440ea6be68" (UID: "e5ef0634-9222-4dd1-be89-2d440ea6be68"). InnerVolumeSpecName "kube-api-access-sv2j8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:25:46.065797 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:46.065704 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sv2j8\" (UniqueName: \"kubernetes.io/projected/e5ef0634-9222-4dd1-be89-2d440ea6be68-kube-api-access-sv2j8\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:25:46.657921 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:46.657885 2566 generic.go:356] "Generic (PLEG): container finished" podID="e5ef0634-9222-4dd1-be89-2d440ea6be68" containerID="6f5f7dc76a39029f415964e6315c0ede29a5425736a7536469fcbb2a033865f8" exitCode=0 Jul 06 18:25:46.658326 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:46.657939 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7vb58" Jul 06 18:25:46.658326 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:46.657966 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7vb58" event={"ID":"e5ef0634-9222-4dd1-be89-2d440ea6be68","Type":"ContainerDied","Data":"6f5f7dc76a39029f415964e6315c0ede29a5425736a7536469fcbb2a033865f8"} Jul 06 18:25:46.658326 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:46.658023 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7vb58" event={"ID":"e5ef0634-9222-4dd1-be89-2d440ea6be68","Type":"ContainerDied","Data":"2e7dc910153fb1ec5f1e412066c7cb4aab89bb8e66e18b86927908d386bd6155"} Jul 06 18:25:46.658326 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:46.658047 2566 scope.go:117] "RemoveContainer" containerID="6f5f7dc76a39029f415964e6315c0ede29a5425736a7536469fcbb2a033865f8" Jul 06 18:25:46.667287 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:46.667269 2566 scope.go:117] "RemoveContainer" containerID="6f5f7dc76a39029f415964e6315c0ede29a5425736a7536469fcbb2a033865f8" Jul 06 18:25:46.667552 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:25:46.667533 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f5f7dc76a39029f415964e6315c0ede29a5425736a7536469fcbb2a033865f8\": container with ID starting with 6f5f7dc76a39029f415964e6315c0ede29a5425736a7536469fcbb2a033865f8 not found: ID does not exist" containerID="6f5f7dc76a39029f415964e6315c0ede29a5425736a7536469fcbb2a033865f8" Jul 06 18:25:46.667600 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:46.667573 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f5f7dc76a39029f415964e6315c0ede29a5425736a7536469fcbb2a033865f8"} err="failed to get container status \"6f5f7dc76a39029f415964e6315c0ede29a5425736a7536469fcbb2a033865f8\": rpc error: code = NotFound desc = could not find container \"6f5f7dc76a39029f415964e6315c0ede29a5425736a7536469fcbb2a033865f8\": container with ID starting with 6f5f7dc76a39029f415964e6315c0ede29a5425736a7536469fcbb2a033865f8 not found: ID does not exist" Jul 06 18:25:46.679416 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:46.679390 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7vb58"] Jul 06 18:25:46.681173 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:46.681152 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7vb58"] Jul 06 18:25:47.849105 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:47.849070 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5ef0634-9222-4dd1-be89-2d440ea6be68" path="/var/lib/kubelet/pods/e5ef0634-9222-4dd1-be89-2d440ea6be68/volumes" Jul 06 18:25:51.682663 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:51.682628 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:25:53.654054 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:53.654016 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-4dc6g" Jul 06 18:25:53.654486 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:53.654083 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-4dc6g" Jul 06 18:25:53.676716 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:53.676692 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-4dc6g" Jul 06 18:25:53.716673 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:53.716646 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-4dc6g" Jul 06 18:25:58.542670 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:58.542632 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs"] Jul 06 18:25:58.543116 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:58.543098 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5ef0634-9222-4dd1-be89-2d440ea6be68" containerName="registry-server" Jul 06 18:25:58.543162 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:58.543121 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5ef0634-9222-4dd1-be89-2d440ea6be68" containerName="registry-server" Jul 06 18:25:58.543220 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:58.543210 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5ef0634-9222-4dd1-be89-2d440ea6be68" containerName="registry-server" Jul 06 18:25:58.547809 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:58.547793 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs" Jul 06 18:25:58.550808 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:58.550787 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jqz8q\"" Jul 06 18:25:58.556097 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:58.556064 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs"] Jul 06 18:25:58.684496 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:58.684456 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clq42\" (UniqueName: \"kubernetes.io/projected/d807c6c2-2ae7-4ed3-b810-eccf138dacc4-kube-api-access-clq42\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs\" (UID: \"d807c6c2-2ae7-4ed3-b810-eccf138dacc4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs" Jul 06 18:25:58.684496 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:58.684509 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d807c6c2-2ae7-4ed3-b810-eccf138dacc4-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs\" (UID: \"d807c6c2-2ae7-4ed3-b810-eccf138dacc4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs" Jul 06 18:25:58.684733 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:58.684545 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d807c6c2-2ae7-4ed3-b810-eccf138dacc4-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs\" (UID: \"d807c6c2-2ae7-4ed3-b810-eccf138dacc4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs" Jul 06 18:25:58.785327 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:58.785289 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clq42\" (UniqueName: \"kubernetes.io/projected/d807c6c2-2ae7-4ed3-b810-eccf138dacc4-kube-api-access-clq42\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs\" (UID: \"d807c6c2-2ae7-4ed3-b810-eccf138dacc4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs" Jul 06 18:25:58.785327 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:58.785335 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d807c6c2-2ae7-4ed3-b810-eccf138dacc4-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs\" (UID: \"d807c6c2-2ae7-4ed3-b810-eccf138dacc4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs" Jul 06 18:25:58.785527 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:58.785368 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d807c6c2-2ae7-4ed3-b810-eccf138dacc4-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs\" (UID: \"d807c6c2-2ae7-4ed3-b810-eccf138dacc4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs" Jul 06 18:25:58.785674 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:58.785656 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d807c6c2-2ae7-4ed3-b810-eccf138dacc4-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs\" (UID: \"d807c6c2-2ae7-4ed3-b810-eccf138dacc4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs" Jul 06 18:25:58.785715 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:58.785678 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d807c6c2-2ae7-4ed3-b810-eccf138dacc4-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs\" (UID: \"d807c6c2-2ae7-4ed3-b810-eccf138dacc4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs" Jul 06 18:25:58.794340 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:58.794277 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clq42\" (UniqueName: \"kubernetes.io/projected/d807c6c2-2ae7-4ed3-b810-eccf138dacc4-kube-api-access-clq42\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs\" (UID: \"d807c6c2-2ae7-4ed3-b810-eccf138dacc4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs" Jul 06 18:25:58.857763 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:58.857726 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs" Jul 06 18:25:58.989413 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:58.989386 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs"] Jul 06 18:25:58.991243 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:25:58.991215 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd807c6c2_2ae7_4ed3_b810_eccf138dacc4.slice/crio-6fbb9397db53e2dbeca7d7944b84bb52dd6014337a0a1e70822689ffc59a0e56 WatchSource:0}: Error finding container 6fbb9397db53e2dbeca7d7944b84bb52dd6014337a0a1e70822689ffc59a0e56: Status 404 returned error can't find the container with id 6fbb9397db53e2dbeca7d7944b84bb52dd6014337a0a1e70822689ffc59a0e56 Jul 06 18:25:59.341659 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:59.341586 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q"] Jul 06 18:25:59.345087 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:59.345069 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q" Jul 06 18:25:59.354156 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:59.354134 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q"] Jul 06 18:25:59.491270 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:59.491236 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9nlv\" (UniqueName: \"kubernetes.io/projected/423da052-df79-441b-9f40-235e7dd852cc-kube-api-access-m9nlv\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q\" (UID: \"423da052-df79-441b-9f40-235e7dd852cc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q" Jul 06 18:25:59.491415 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:59.491280 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/423da052-df79-441b-9f40-235e7dd852cc-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q\" (UID: \"423da052-df79-441b-9f40-235e7dd852cc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q" Jul 06 18:25:59.491415 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:59.491308 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/423da052-df79-441b-9f40-235e7dd852cc-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q\" (UID: \"423da052-df79-441b-9f40-235e7dd852cc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q" Jul 06 18:25:59.592843 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:59.592741 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9nlv\" (UniqueName: \"kubernetes.io/projected/423da052-df79-441b-9f40-235e7dd852cc-kube-api-access-m9nlv\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q\" (UID: \"423da052-df79-441b-9f40-235e7dd852cc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q" Jul 06 18:25:59.592843 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:59.592797 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/423da052-df79-441b-9f40-235e7dd852cc-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q\" (UID: \"423da052-df79-441b-9f40-235e7dd852cc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q" Jul 06 18:25:59.593356 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:59.592911 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/423da052-df79-441b-9f40-235e7dd852cc-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q\" (UID: \"423da052-df79-441b-9f40-235e7dd852cc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q" Jul 06 18:25:59.593356 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:59.593117 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/423da052-df79-441b-9f40-235e7dd852cc-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q\" (UID: \"423da052-df79-441b-9f40-235e7dd852cc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q" Jul 06 18:25:59.593356 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:59.593251 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/423da052-df79-441b-9f40-235e7dd852cc-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q\" (UID: \"423da052-df79-441b-9f40-235e7dd852cc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q" Jul 06 18:25:59.601435 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:59.601405 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9nlv\" (UniqueName: \"kubernetes.io/projected/423da052-df79-441b-9f40-235e7dd852cc-kube-api-access-m9nlv\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q\" (UID: \"423da052-df79-441b-9f40-235e7dd852cc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q" Jul 06 18:25:59.654965 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:59.654926 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q" Jul 06 18:25:59.724841 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:59.724806 2566 generic.go:356] "Generic (PLEG): container finished" podID="d807c6c2-2ae7-4ed3-b810-eccf138dacc4" containerID="ccab0f21c287991193cc9b3aba0b43a0954b79c2ff4000dd1016048df83ee6b5" exitCode=0 Jul 06 18:25:59.725034 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:59.724912 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs" event={"ID":"d807c6c2-2ae7-4ed3-b810-eccf138dacc4","Type":"ContainerDied","Data":"ccab0f21c287991193cc9b3aba0b43a0954b79c2ff4000dd1016048df83ee6b5"} Jul 06 18:25:59.725034 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:59.724976 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs" event={"ID":"d807c6c2-2ae7-4ed3-b810-eccf138dacc4","Type":"ContainerStarted","Data":"6fbb9397db53e2dbeca7d7944b84bb52dd6014337a0a1e70822689ffc59a0e56"} Jul 06 18:25:59.793564 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:59.793537 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q"] Jul 06 18:25:59.795225 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:25:59.795197 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod423da052_df79_441b_9f40_235e7dd852cc.slice/crio-4f622056ae2280c30d534061a1b0004f38840697fb80b92f71453bd8ca231fa4 WatchSource:0}: Error finding container 4f622056ae2280c30d534061a1b0004f38840697fb80b92f71453bd8ca231fa4: Status 404 returned error can't find the container with id 4f622056ae2280c30d534061a1b0004f38840697fb80b92f71453bd8ca231fa4 Jul 06 18:25:59.943126 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:59.943094 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4"] Jul 06 18:25:59.946722 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:59.946701 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4" Jul 06 18:25:59.954452 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:25:59.954419 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4"] Jul 06 18:26:00.097964 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.097932 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30418921-3908-47df-8ea2-b5ff2737ba45-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4\" (UID: \"30418921-3908-47df-8ea2-b5ff2737ba45\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4" Jul 06 18:26:00.098126 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.098015 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4jfn\" (UniqueName: \"kubernetes.io/projected/30418921-3908-47df-8ea2-b5ff2737ba45-kube-api-access-q4jfn\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4\" (UID: \"30418921-3908-47df-8ea2-b5ff2737ba45\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4" Jul 06 18:26:00.098126 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.098042 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30418921-3908-47df-8ea2-b5ff2737ba45-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4\" (UID: \"30418921-3908-47df-8ea2-b5ff2737ba45\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4" Jul 06 18:26:00.198850 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.198824 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30418921-3908-47df-8ea2-b5ff2737ba45-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4\" (UID: \"30418921-3908-47df-8ea2-b5ff2737ba45\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4" Jul 06 18:26:00.198972 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.198928 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4jfn\" (UniqueName: \"kubernetes.io/projected/30418921-3908-47df-8ea2-b5ff2737ba45-kube-api-access-q4jfn\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4\" (UID: \"30418921-3908-47df-8ea2-b5ff2737ba45\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4" Jul 06 18:26:00.199038 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.198967 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30418921-3908-47df-8ea2-b5ff2737ba45-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4\" (UID: \"30418921-3908-47df-8ea2-b5ff2737ba45\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4" Jul 06 18:26:00.199161 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.199141 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30418921-3908-47df-8ea2-b5ff2737ba45-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4\" (UID: \"30418921-3908-47df-8ea2-b5ff2737ba45\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4" Jul 06 18:26:00.199281 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.199264 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30418921-3908-47df-8ea2-b5ff2737ba45-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4\" (UID: \"30418921-3908-47df-8ea2-b5ff2737ba45\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4" Jul 06 18:26:00.208517 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.208489 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4jfn\" (UniqueName: \"kubernetes.io/projected/30418921-3908-47df-8ea2-b5ff2737ba45-kube-api-access-q4jfn\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4\" (UID: \"30418921-3908-47df-8ea2-b5ff2737ba45\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4" Jul 06 18:26:00.277098 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.277072 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4" Jul 06 18:26:00.346078 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.346049 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg"] Jul 06 18:26:00.354932 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.354896 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg" Jul 06 18:26:00.358626 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.358585 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg"] Jul 06 18:26:00.421035 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.421007 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4"] Jul 06 18:26:00.422656 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:26:00.422629 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30418921_3908_47df_8ea2_b5ff2737ba45.slice/crio-22c0e695730c897925918e6047a9bbd2d95f631d8d232d3b36c0c4de6429c35d WatchSource:0}: Error finding container 22c0e695730c897925918e6047a9bbd2d95f631d8d232d3b36c0c4de6429c35d: Status 404 returned error can't find the container with id 22c0e695730c897925918e6047a9bbd2d95f631d8d232d3b36c0c4de6429c35d Jul 06 18:26:00.502183 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.502148 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9a4973c-1b3d-4742-b736-0866b5383695-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg\" (UID: \"e9a4973c-1b3d-4742-b736-0866b5383695\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg" Jul 06 18:26:00.502269 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.502209 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9a4973c-1b3d-4742-b736-0866b5383695-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg\" (UID: \"e9a4973c-1b3d-4742-b736-0866b5383695\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg" Jul 06 18:26:00.502269 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.502248 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxt5h\" (UniqueName: \"kubernetes.io/projected/e9a4973c-1b3d-4742-b736-0866b5383695-kube-api-access-fxt5h\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg\" (UID: \"e9a4973c-1b3d-4742-b736-0866b5383695\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg" Jul 06 18:26:00.602945 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.602914 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9a4973c-1b3d-4742-b736-0866b5383695-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg\" (UID: \"e9a4973c-1b3d-4742-b736-0866b5383695\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg" Jul 06 18:26:00.603376 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.602963 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9a4973c-1b3d-4742-b736-0866b5383695-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg\" (UID: \"e9a4973c-1b3d-4742-b736-0866b5383695\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg" Jul 06 18:26:00.603376 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.603008 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxt5h\" (UniqueName: \"kubernetes.io/projected/e9a4973c-1b3d-4742-b736-0866b5383695-kube-api-access-fxt5h\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg\" (UID: \"e9a4973c-1b3d-4742-b736-0866b5383695\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg" Jul 06 18:26:00.603376 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.603356 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9a4973c-1b3d-4742-b736-0866b5383695-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg\" (UID: \"e9a4973c-1b3d-4742-b736-0866b5383695\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg" Jul 06 18:26:00.603684 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.603395 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9a4973c-1b3d-4742-b736-0866b5383695-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg\" (UID: \"e9a4973c-1b3d-4742-b736-0866b5383695\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg" Jul 06 18:26:00.611793 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.611769 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxt5h\" (UniqueName: \"kubernetes.io/projected/e9a4973c-1b3d-4742-b736-0866b5383695-kube-api-access-fxt5h\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg\" (UID: \"e9a4973c-1b3d-4742-b736-0866b5383695\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg" Jul 06 18:26:00.666945 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.666910 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg" Jul 06 18:26:00.732582 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.732548 2566 generic.go:356] "Generic (PLEG): container finished" podID="d807c6c2-2ae7-4ed3-b810-eccf138dacc4" containerID="12f1b0b9302a35963f03453adbd475c26947c6c4ad7486c8770637a735675cd5" exitCode=0 Jul 06 18:26:00.732716 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.732628 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs" event={"ID":"d807c6c2-2ae7-4ed3-b810-eccf138dacc4","Type":"ContainerDied","Data":"12f1b0b9302a35963f03453adbd475c26947c6c4ad7486c8770637a735675cd5"} Jul 06 18:26:00.734358 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.734313 2566 generic.go:356] "Generic (PLEG): container finished" podID="30418921-3908-47df-8ea2-b5ff2737ba45" containerID="ab9c29e3e5adc6afe79544f7c61631e47f04f0bfe0d2bca93f74e0828e7b865a" exitCode=0 Jul 06 18:26:00.734430 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.734397 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4" event={"ID":"30418921-3908-47df-8ea2-b5ff2737ba45","Type":"ContainerDied","Data":"ab9c29e3e5adc6afe79544f7c61631e47f04f0bfe0d2bca93f74e0828e7b865a"} Jul 06 18:26:00.734490 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.734440 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4" event={"ID":"30418921-3908-47df-8ea2-b5ff2737ba45","Type":"ContainerStarted","Data":"22c0e695730c897925918e6047a9bbd2d95f631d8d232d3b36c0c4de6429c35d"} Jul 06 18:26:00.736029 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.736004 2566 generic.go:356] "Generic (PLEG): container finished" podID="423da052-df79-441b-9f40-235e7dd852cc" containerID="f00b274d8ae463088faa1d9753ef483ec7864e673f93c1e76762651bcf7cd6eb" exitCode=0 Jul 06 18:26:00.736141 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.736063 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q" event={"ID":"423da052-df79-441b-9f40-235e7dd852cc","Type":"ContainerDied","Data":"f00b274d8ae463088faa1d9753ef483ec7864e673f93c1e76762651bcf7cd6eb"} Jul 06 18:26:00.736141 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.736096 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q" event={"ID":"423da052-df79-441b-9f40-235e7dd852cc","Type":"ContainerStarted","Data":"4f622056ae2280c30d534061a1b0004f38840697fb80b92f71453bd8ca231fa4"} Jul 06 18:26:00.809398 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:00.809369 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg"] Jul 06 18:26:00.811981 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:26:00.811956 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9a4973c_1b3d_4742_b736_0866b5383695.slice/crio-b1cfe59b0c877931190e0e16137d9234d942b1fec765dcfe1853d3542c91e661 WatchSource:0}: Error finding container b1cfe59b0c877931190e0e16137d9234d942b1fec765dcfe1853d3542c91e661: Status 404 returned error can't find the container with id b1cfe59b0c877931190e0e16137d9234d942b1fec765dcfe1853d3542c91e661 Jul 06 18:26:01.742632 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:01.742533 2566 generic.go:356] "Generic (PLEG): container finished" podID="30418921-3908-47df-8ea2-b5ff2737ba45" containerID="5afff985ef5a0a053305b22af3ed8b1dd3fed3e59813beeb08e174c8c23c66c4" exitCode=0 Jul 06 18:26:01.743070 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:01.742629 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4" event={"ID":"30418921-3908-47df-8ea2-b5ff2737ba45","Type":"ContainerDied","Data":"5afff985ef5a0a053305b22af3ed8b1dd3fed3e59813beeb08e174c8c23c66c4"} Jul 06 18:26:01.744656 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:01.744628 2566 generic.go:356] "Generic (PLEG): container finished" podID="423da052-df79-441b-9f40-235e7dd852cc" containerID="ab96f08876ec1db5b15dcc64a2d6e3c264c3d30289f4d5dd31cc7ffc57e6e764" exitCode=0 Jul 06 18:26:01.744736 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:01.744657 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q" event={"ID":"423da052-df79-441b-9f40-235e7dd852cc","Type":"ContainerDied","Data":"ab96f08876ec1db5b15dcc64a2d6e3c264c3d30289f4d5dd31cc7ffc57e6e764"} Jul 06 18:26:01.746315 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:01.746293 2566 generic.go:356] "Generic (PLEG): container finished" podID="e9a4973c-1b3d-4742-b736-0866b5383695" containerID="9498c62d099d94875e2e73d5df06123e790daba0bedca25ebd10f1b5a7459673" exitCode=0 Jul 06 18:26:01.746393 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:01.746364 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg" event={"ID":"e9a4973c-1b3d-4742-b736-0866b5383695","Type":"ContainerDied","Data":"9498c62d099d94875e2e73d5df06123e790daba0bedca25ebd10f1b5a7459673"} Jul 06 18:26:01.746439 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:01.746398 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg" event={"ID":"e9a4973c-1b3d-4742-b736-0866b5383695","Type":"ContainerStarted","Data":"b1cfe59b0c877931190e0e16137d9234d942b1fec765dcfe1853d3542c91e661"} Jul 06 18:26:01.748795 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:01.748762 2566 generic.go:356] "Generic (PLEG): container finished" podID="d807c6c2-2ae7-4ed3-b810-eccf138dacc4" containerID="e91b9a62d9afe6fffef5804bc876c3d65f6e77cfbb78f8ff7153d0f4bc498677" exitCode=0 Jul 06 18:26:01.748895 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:01.748807 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs" event={"ID":"d807c6c2-2ae7-4ed3-b810-eccf138dacc4","Type":"ContainerDied","Data":"e91b9a62d9afe6fffef5804bc876c3d65f6e77cfbb78f8ff7153d0f4bc498677"} Jul 06 18:26:02.755779 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:02.755690 2566 generic.go:356] "Generic (PLEG): container finished" podID="e9a4973c-1b3d-4742-b736-0866b5383695" containerID="aa7fd8abb343b7b493e6f5c2a87dfb151776048bbd1e2a15787557ce22104ba5" exitCode=0 Jul 06 18:26:02.756215 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:02.755774 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg" event={"ID":"e9a4973c-1b3d-4742-b736-0866b5383695","Type":"ContainerDied","Data":"aa7fd8abb343b7b493e6f5c2a87dfb151776048bbd1e2a15787557ce22104ba5"} Jul 06 18:26:02.758058 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:02.758034 2566 generic.go:356] "Generic (PLEG): container finished" podID="30418921-3908-47df-8ea2-b5ff2737ba45" containerID="1eac98cc0a274aaf749ff95ffe677440a68a6f92abb71142f927cfe093caab30" exitCode=0 Jul 06 18:26:02.758151 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:02.758117 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4" event={"ID":"30418921-3908-47df-8ea2-b5ff2737ba45","Type":"ContainerDied","Data":"1eac98cc0a274aaf749ff95ffe677440a68a6f92abb71142f927cfe093caab30"} Jul 06 18:26:02.760115 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:02.760094 2566 generic.go:356] "Generic (PLEG): container finished" podID="423da052-df79-441b-9f40-235e7dd852cc" containerID="bf5e4d371ef096016d3e7e09dba5904e7058972802723c4fa7a7a8d879b87a4f" exitCode=0 Jul 06 18:26:02.760192 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:02.760173 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q" event={"ID":"423da052-df79-441b-9f40-235e7dd852cc","Type":"ContainerDied","Data":"bf5e4d371ef096016d3e7e09dba5904e7058972802723c4fa7a7a8d879b87a4f"} Jul 06 18:26:02.911528 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:02.911487 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs" Jul 06 18:26:03.027145 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:03.027058 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d807c6c2-2ae7-4ed3-b810-eccf138dacc4-util\") pod \"d807c6c2-2ae7-4ed3-b810-eccf138dacc4\" (UID: \"d807c6c2-2ae7-4ed3-b810-eccf138dacc4\") " Jul 06 18:26:03.027145 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:03.027141 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d807c6c2-2ae7-4ed3-b810-eccf138dacc4-bundle\") pod \"d807c6c2-2ae7-4ed3-b810-eccf138dacc4\" (UID: \"d807c6c2-2ae7-4ed3-b810-eccf138dacc4\") " Jul 06 18:26:03.027356 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:03.027179 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clq42\" (UniqueName: \"kubernetes.io/projected/d807c6c2-2ae7-4ed3-b810-eccf138dacc4-kube-api-access-clq42\") pod \"d807c6c2-2ae7-4ed3-b810-eccf138dacc4\" (UID: \"d807c6c2-2ae7-4ed3-b810-eccf138dacc4\") " Jul 06 18:26:03.027644 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:03.027621 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d807c6c2-2ae7-4ed3-b810-eccf138dacc4-bundle" (OuterVolumeSpecName: "bundle") pod "d807c6c2-2ae7-4ed3-b810-eccf138dacc4" (UID: "d807c6c2-2ae7-4ed3-b810-eccf138dacc4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:26:03.029919 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:03.029888 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d807c6c2-2ae7-4ed3-b810-eccf138dacc4-kube-api-access-clq42" (OuterVolumeSpecName: "kube-api-access-clq42") pod "d807c6c2-2ae7-4ed3-b810-eccf138dacc4" (UID: "d807c6c2-2ae7-4ed3-b810-eccf138dacc4"). InnerVolumeSpecName "kube-api-access-clq42". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:26:03.032779 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:03.032740 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d807c6c2-2ae7-4ed3-b810-eccf138dacc4-util" (OuterVolumeSpecName: "util") pod "d807c6c2-2ae7-4ed3-b810-eccf138dacc4" (UID: "d807c6c2-2ae7-4ed3-b810-eccf138dacc4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:26:03.127846 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:03.127797 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d807c6c2-2ae7-4ed3-b810-eccf138dacc4-bundle\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:26:03.127846 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:03.127843 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clq42\" (UniqueName: \"kubernetes.io/projected/d807c6c2-2ae7-4ed3-b810-eccf138dacc4-kube-api-access-clq42\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:26:03.127846 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:03.127855 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d807c6c2-2ae7-4ed3-b810-eccf138dacc4-util\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:26:03.248577 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:03.248543 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:03.350306 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:03.350209 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:03.768014 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:03.767977 2566 generic.go:356] "Generic (PLEG): container finished" podID="e9a4973c-1b3d-4742-b736-0866b5383695" containerID="f2874db31aefcf2f75a024e42754922c64405eef988ec83eb62227ef4a5b506c" exitCode=0 Jul 06 18:26:03.768416 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:03.768054 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg" event={"ID":"e9a4973c-1b3d-4742-b736-0866b5383695","Type":"ContainerDied","Data":"f2874db31aefcf2f75a024e42754922c64405eef988ec83eb62227ef4a5b506c"} Jul 06 18:26:03.769981 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:03.769962 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs" Jul 06 18:26:03.770079 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:03.769992 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs" event={"ID":"d807c6c2-2ae7-4ed3-b810-eccf138dacc4","Type":"ContainerDied","Data":"6fbb9397db53e2dbeca7d7944b84bb52dd6014337a0a1e70822689ffc59a0e56"} Jul 06 18:26:03.770079 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:03.770030 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6fbb9397db53e2dbeca7d7944b84bb52dd6014337a0a1e70822689ffc59a0e56" Jul 06 18:26:03.929967 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:03.929945 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q" Jul 06 18:26:03.932978 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:03.932956 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4" Jul 06 18:26:04.034555 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:04.034458 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30418921-3908-47df-8ea2-b5ff2737ba45-util\") pod \"30418921-3908-47df-8ea2-b5ff2737ba45\" (UID: \"30418921-3908-47df-8ea2-b5ff2737ba45\") " Jul 06 18:26:04.034555 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:04.034503 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/423da052-df79-441b-9f40-235e7dd852cc-util\") pod \"423da052-df79-441b-9f40-235e7dd852cc\" (UID: \"423da052-df79-441b-9f40-235e7dd852cc\") " Jul 06 18:26:04.034555 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:04.034542 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30418921-3908-47df-8ea2-b5ff2737ba45-bundle\") pod \"30418921-3908-47df-8ea2-b5ff2737ba45\" (UID: \"30418921-3908-47df-8ea2-b5ff2737ba45\") " Jul 06 18:26:04.034892 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:04.034566 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4jfn\" (UniqueName: \"kubernetes.io/projected/30418921-3908-47df-8ea2-b5ff2737ba45-kube-api-access-q4jfn\") pod \"30418921-3908-47df-8ea2-b5ff2737ba45\" (UID: \"30418921-3908-47df-8ea2-b5ff2737ba45\") " Jul 06 18:26:04.034892 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:04.034598 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9nlv\" (UniqueName: \"kubernetes.io/projected/423da052-df79-441b-9f40-235e7dd852cc-kube-api-access-m9nlv\") pod \"423da052-df79-441b-9f40-235e7dd852cc\" (UID: \"423da052-df79-441b-9f40-235e7dd852cc\") " Jul 06 18:26:04.034892 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:04.034709 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/423da052-df79-441b-9f40-235e7dd852cc-bundle\") pod \"423da052-df79-441b-9f40-235e7dd852cc\" (UID: \"423da052-df79-441b-9f40-235e7dd852cc\") " Jul 06 18:26:04.035237 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:04.035207 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30418921-3908-47df-8ea2-b5ff2737ba45-bundle" (OuterVolumeSpecName: "bundle") pod "30418921-3908-47df-8ea2-b5ff2737ba45" (UID: "30418921-3908-47df-8ea2-b5ff2737ba45"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:26:04.035505 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:04.035443 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/423da052-df79-441b-9f40-235e7dd852cc-bundle" (OuterVolumeSpecName: "bundle") pod "423da052-df79-441b-9f40-235e7dd852cc" (UID: "423da052-df79-441b-9f40-235e7dd852cc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:26:04.036957 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:04.036925 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/423da052-df79-441b-9f40-235e7dd852cc-kube-api-access-m9nlv" (OuterVolumeSpecName: "kube-api-access-m9nlv") pod "423da052-df79-441b-9f40-235e7dd852cc" (UID: "423da052-df79-441b-9f40-235e7dd852cc"). InnerVolumeSpecName "kube-api-access-m9nlv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:26:04.037136 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:04.037114 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30418921-3908-47df-8ea2-b5ff2737ba45-kube-api-access-q4jfn" (OuterVolumeSpecName: "kube-api-access-q4jfn") pod "30418921-3908-47df-8ea2-b5ff2737ba45" (UID: "30418921-3908-47df-8ea2-b5ff2737ba45"). InnerVolumeSpecName "kube-api-access-q4jfn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:26:04.039951 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:04.039915 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30418921-3908-47df-8ea2-b5ff2737ba45-util" (OuterVolumeSpecName: "util") pod "30418921-3908-47df-8ea2-b5ff2737ba45" (UID: "30418921-3908-47df-8ea2-b5ff2737ba45"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:26:04.041273 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:04.041249 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/423da052-df79-441b-9f40-235e7dd852cc-util" (OuterVolumeSpecName: "util") pod "423da052-df79-441b-9f40-235e7dd852cc" (UID: "423da052-df79-441b-9f40-235e7dd852cc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:26:04.136071 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:04.136029 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/423da052-df79-441b-9f40-235e7dd852cc-bundle\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:26:04.136071 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:04.136063 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30418921-3908-47df-8ea2-b5ff2737ba45-util\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:26:04.136071 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:04.136074 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/423da052-df79-441b-9f40-235e7dd852cc-util\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:26:04.136071 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:04.136084 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30418921-3908-47df-8ea2-b5ff2737ba45-bundle\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:26:04.136335 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:04.136096 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4jfn\" (UniqueName: \"kubernetes.io/projected/30418921-3908-47df-8ea2-b5ff2737ba45-kube-api-access-q4jfn\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:26:04.136335 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:04.136109 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9nlv\" (UniqueName: \"kubernetes.io/projected/423da052-df79-441b-9f40-235e7dd852cc-kube-api-access-m9nlv\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:26:04.777111 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:04.777082 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4" Jul 06 18:26:04.777560 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:04.777078 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4" event={"ID":"30418921-3908-47df-8ea2-b5ff2737ba45","Type":"ContainerDied","Data":"22c0e695730c897925918e6047a9bbd2d95f631d8d232d3b36c0c4de6429c35d"} Jul 06 18:26:04.777560 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:04.777203 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="22c0e695730c897925918e6047a9bbd2d95f631d8d232d3b36c0c4de6429c35d" Jul 06 18:26:04.778994 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:04.778974 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q" Jul 06 18:26:04.779118 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:04.779008 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q" event={"ID":"423da052-df79-441b-9f40-235e7dd852cc","Type":"ContainerDied","Data":"4f622056ae2280c30d534061a1b0004f38840697fb80b92f71453bd8ca231fa4"} Jul 06 18:26:04.779118 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:04.779044 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4f622056ae2280c30d534061a1b0004f38840697fb80b92f71453bd8ca231fa4" Jul 06 18:26:04.898364 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:04.898335 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg" Jul 06 18:26:05.043975 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:05.043863 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9a4973c-1b3d-4742-b736-0866b5383695-util\") pod \"e9a4973c-1b3d-4742-b736-0866b5383695\" (UID: \"e9a4973c-1b3d-4742-b736-0866b5383695\") " Jul 06 18:26:05.043975 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:05.043942 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxt5h\" (UniqueName: \"kubernetes.io/projected/e9a4973c-1b3d-4742-b736-0866b5383695-kube-api-access-fxt5h\") pod \"e9a4973c-1b3d-4742-b736-0866b5383695\" (UID: \"e9a4973c-1b3d-4742-b736-0866b5383695\") " Jul 06 18:26:05.044202 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:05.043994 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9a4973c-1b3d-4742-b736-0866b5383695-bundle\") pod \"e9a4973c-1b3d-4742-b736-0866b5383695\" (UID: \"e9a4973c-1b3d-4742-b736-0866b5383695\") " Jul 06 18:26:05.044582 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:05.044552 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9a4973c-1b3d-4742-b736-0866b5383695-bundle" (OuterVolumeSpecName: "bundle") pod "e9a4973c-1b3d-4742-b736-0866b5383695" (UID: "e9a4973c-1b3d-4742-b736-0866b5383695"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:26:05.046194 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:05.046164 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9a4973c-1b3d-4742-b736-0866b5383695-kube-api-access-fxt5h" (OuterVolumeSpecName: "kube-api-access-fxt5h") pod "e9a4973c-1b3d-4742-b736-0866b5383695" (UID: "e9a4973c-1b3d-4742-b736-0866b5383695"). InnerVolumeSpecName "kube-api-access-fxt5h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:26:05.049089 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:05.049053 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9a4973c-1b3d-4742-b736-0866b5383695-util" (OuterVolumeSpecName: "util") pod "e9a4973c-1b3d-4742-b736-0866b5383695" (UID: "e9a4973c-1b3d-4742-b736-0866b5383695"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:26:05.144958 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:05.144906 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9a4973c-1b3d-4742-b736-0866b5383695-util\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:26:05.144958 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:05.144952 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxt5h\" (UniqueName: \"kubernetes.io/projected/e9a4973c-1b3d-4742-b736-0866b5383695-kube-api-access-fxt5h\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:26:05.144958 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:05.144963 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9a4973c-1b3d-4742-b736-0866b5383695-bundle\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:26:05.786154 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:05.786124 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg" Jul 06 18:26:05.786556 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:05.786151 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg" event={"ID":"e9a4973c-1b3d-4742-b736-0866b5383695","Type":"ContainerDied","Data":"b1cfe59b0c877931190e0e16137d9234d942b1fec765dcfe1853d3542c91e661"} Jul 06 18:26:05.786556 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:05.786243 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b1cfe59b0c877931190e0e16137d9234d942b1fec765dcfe1853d3542c91e661" Jul 06 18:26:12.071735 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:12.071698 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:15.518065 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:15.518031 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:16.781941 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.781902 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gtbr8"] Jul 06 18:26:16.782318 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.782291 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d807c6c2-2ae7-4ed3-b810-eccf138dacc4" containerName="util" Jul 06 18:26:16.782318 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.782304 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d807c6c2-2ae7-4ed3-b810-eccf138dacc4" containerName="util" Jul 06 18:26:16.782318 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.782312 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30418921-3908-47df-8ea2-b5ff2737ba45" containerName="extract" Jul 06 18:26:16.782318 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.782319 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="30418921-3908-47df-8ea2-b5ff2737ba45" containerName="extract" Jul 06 18:26:16.782454 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.782325 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30418921-3908-47df-8ea2-b5ff2737ba45" containerName="pull" Jul 06 18:26:16.782454 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.782332 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="30418921-3908-47df-8ea2-b5ff2737ba45" containerName="pull" Jul 06 18:26:16.782454 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.782345 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d807c6c2-2ae7-4ed3-b810-eccf138dacc4" containerName="extract" Jul 06 18:26:16.782454 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.782350 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d807c6c2-2ae7-4ed3-b810-eccf138dacc4" containerName="extract" Jul 06 18:26:16.782454 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.782362 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d807c6c2-2ae7-4ed3-b810-eccf138dacc4" containerName="pull" Jul 06 18:26:16.782454 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.782367 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d807c6c2-2ae7-4ed3-b810-eccf138dacc4" containerName="pull" Jul 06 18:26:16.782454 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.782372 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9a4973c-1b3d-4742-b736-0866b5383695" containerName="pull" Jul 06 18:26:16.782454 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.782377 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9a4973c-1b3d-4742-b736-0866b5383695" containerName="pull" Jul 06 18:26:16.782454 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.782387 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="423da052-df79-441b-9f40-235e7dd852cc" containerName="extract" Jul 06 18:26:16.782454 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.782392 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="423da052-df79-441b-9f40-235e7dd852cc" containerName="extract" Jul 06 18:26:16.782454 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.782401 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="423da052-df79-441b-9f40-235e7dd852cc" containerName="util" Jul 06 18:26:16.782454 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.782407 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="423da052-df79-441b-9f40-235e7dd852cc" containerName="util" Jul 06 18:26:16.782454 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.782418 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30418921-3908-47df-8ea2-b5ff2737ba45" containerName="util" Jul 06 18:26:16.782454 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.782425 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="30418921-3908-47df-8ea2-b5ff2737ba45" containerName="util" Jul 06 18:26:16.782454 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.782430 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9a4973c-1b3d-4742-b736-0866b5383695" containerName="util" Jul 06 18:26:16.782454 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.782435 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9a4973c-1b3d-4742-b736-0866b5383695" containerName="util" Jul 06 18:26:16.782454 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.782441 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9a4973c-1b3d-4742-b736-0866b5383695" containerName="extract" Jul 06 18:26:16.782454 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.782446 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9a4973c-1b3d-4742-b736-0866b5383695" containerName="extract" Jul 06 18:26:16.782454 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.782455 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="423da052-df79-441b-9f40-235e7dd852cc" containerName="pull" Jul 06 18:26:16.782454 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.782461 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="423da052-df79-441b-9f40-235e7dd852cc" containerName="pull" Jul 06 18:26:16.783069 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.782529 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="30418921-3908-47df-8ea2-b5ff2737ba45" containerName="extract" Jul 06 18:26:16.783069 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.782542 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9a4973c-1b3d-4742-b736-0866b5383695" containerName="extract" Jul 06 18:26:16.783069 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.782548 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="423da052-df79-441b-9f40-235e7dd852cc" containerName="extract" Jul 06 18:26:16.783069 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.782555 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d807c6c2-2ae7-4ed3-b810-eccf138dacc4" containerName="extract" Jul 06 18:26:16.786993 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.786978 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gtbr8" Jul 06 18:26:16.789990 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.789968 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9gb28\"" Jul 06 18:26:16.799086 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.799066 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gtbr8"] Jul 06 18:26:16.854139 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.854106 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rwhj\" (UniqueName: \"kubernetes.io/projected/0c313f99-6b41-44dc-9180-f5e686dd4e69-kube-api-access-9rwhj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gtbr8\" (UID: \"0c313f99-6b41-44dc-9180-f5e686dd4e69\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gtbr8" Jul 06 18:26:16.854315 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.854154 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c313f99-6b41-44dc-9180-f5e686dd4e69-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gtbr8\" (UID: \"0c313f99-6b41-44dc-9180-f5e686dd4e69\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gtbr8" Jul 06 18:26:16.955448 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.955409 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c313f99-6b41-44dc-9180-f5e686dd4e69-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gtbr8\" (UID: \"0c313f99-6b41-44dc-9180-f5e686dd4e69\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gtbr8" Jul 06 18:26:16.955631 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.955492 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rwhj\" (UniqueName: \"kubernetes.io/projected/0c313f99-6b41-44dc-9180-f5e686dd4e69-kube-api-access-9rwhj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gtbr8\" (UID: \"0c313f99-6b41-44dc-9180-f5e686dd4e69\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gtbr8" Jul 06 18:26:16.955793 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.955772 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c313f99-6b41-44dc-9180-f5e686dd4e69-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gtbr8\" (UID: \"0c313f99-6b41-44dc-9180-f5e686dd4e69\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gtbr8" Jul 06 18:26:16.964266 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:16.964242 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rwhj\" (UniqueName: \"kubernetes.io/projected/0c313f99-6b41-44dc-9180-f5e686dd4e69-kube-api-access-9rwhj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gtbr8\" (UID: \"0c313f99-6b41-44dc-9180-f5e686dd4e69\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gtbr8" Jul 06 18:26:17.097583 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:17.097492 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gtbr8" Jul 06 18:26:17.178008 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:17.177978 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:17.248037 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:17.248006 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gtbr8"] Jul 06 18:26:17.249337 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:26:17.249311 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c313f99_6b41_44dc_9180_f5e686dd4e69.slice/crio-a33ffe1be35fe3db5ae95ebc06b3d69ec9d24b6ce30b759aa2613e05df38e8ff WatchSource:0}: Error finding container a33ffe1be35fe3db5ae95ebc06b3d69ec9d24b6ce30b759aa2613e05df38e8ff: Status 404 returned error can't find the container with id a33ffe1be35fe3db5ae95ebc06b3d69ec9d24b6ce30b759aa2613e05df38e8ff Jul 06 18:26:17.480065 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:17.480026 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:17.851423 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:17.851330 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gtbr8" event={"ID":"0c313f99-6b41-44dc-9180-f5e686dd4e69","Type":"ContainerStarted","Data":"a33ffe1be35fe3db5ae95ebc06b3d69ec9d24b6ce30b759aa2613e05df38e8ff"} Jul 06 18:26:18.555549 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:18.555515 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:19.491657 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:19.491617 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-qmdl6"] Jul 06 18:26:19.499836 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:19.499805 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qmdl6" Jul 06 18:26:19.505370 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:19.505346 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-gbq2k\"" Jul 06 18:26:19.512589 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:19.512566 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-qmdl6"] Jul 06 18:26:19.580651 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:19.580583 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d726\" (UniqueName: \"kubernetes.io/projected/66bc78d7-1b98-4795-9097-641e96f71ba1-kube-api-access-5d726\") pod \"authorino-operator-75cc8d5cb7-qmdl6\" (UID: \"66bc78d7-1b98-4795-9097-641e96f71ba1\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-qmdl6" Jul 06 18:26:19.681966 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:19.681925 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d726\" (UniqueName: \"kubernetes.io/projected/66bc78d7-1b98-4795-9097-641e96f71ba1-kube-api-access-5d726\") pod \"authorino-operator-75cc8d5cb7-qmdl6\" (UID: \"66bc78d7-1b98-4795-9097-641e96f71ba1\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-qmdl6" Jul 06 18:26:19.690705 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:19.690674 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d726\" (UniqueName: \"kubernetes.io/projected/66bc78d7-1b98-4795-9097-641e96f71ba1-kube-api-access-5d726\") pod \"authorino-operator-75cc8d5cb7-qmdl6\" (UID: \"66bc78d7-1b98-4795-9097-641e96f71ba1\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-qmdl6" Jul 06 18:26:19.813800 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:19.813703 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qmdl6" Jul 06 18:26:19.981778 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:19.981743 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-qmdl6"] Jul 06 18:26:19.983039 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:26:19.983011 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66bc78d7_1b98_4795_9097_641e96f71ba1.slice/crio-11af2f657275281f1e006f4a7a05eaa434ccdef0eb71e3d656cc57ab2a3c14fe WatchSource:0}: Error finding container 11af2f657275281f1e006f4a7a05eaa434ccdef0eb71e3d656cc57ab2a3c14fe: Status 404 returned error can't find the container with id 11af2f657275281f1e006f4a7a05eaa434ccdef0eb71e3d656cc57ab2a3c14fe Jul 06 18:26:20.107122 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:20.107027 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:20.200418 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:20.200385 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:20.874739 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:20.874699 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qmdl6" event={"ID":"66bc78d7-1b98-4795-9097-641e96f71ba1","Type":"ContainerStarted","Data":"11af2f657275281f1e006f4a7a05eaa434ccdef0eb71e3d656cc57ab2a3c14fe"} Jul 06 18:26:22.289806 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:22.289765 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:22.474020 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:22.473975 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f6988b95b-7w4kf"] Jul 06 18:26:22.478159 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:22.478122 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f6988b95b-7w4kf" Jul 06 18:26:22.499878 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:22.499826 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f6988b95b-7w4kf"] Jul 06 18:26:22.514123 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:22.514090 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8d33609e-12bd-4b5f-ba47-7bb64348eb3e-console-oauth-config\") pod \"console-5f6988b95b-7w4kf\" (UID: \"8d33609e-12bd-4b5f-ba47-7bb64348eb3e\") " pod="openshift-console/console-5f6988b95b-7w4kf" Jul 06 18:26:22.514292 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:22.514126 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d33609e-12bd-4b5f-ba47-7bb64348eb3e-trusted-ca-bundle\") pod \"console-5f6988b95b-7w4kf\" (UID: \"8d33609e-12bd-4b5f-ba47-7bb64348eb3e\") " pod="openshift-console/console-5f6988b95b-7w4kf" Jul 06 18:26:22.514292 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:22.514167 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8d33609e-12bd-4b5f-ba47-7bb64348eb3e-oauth-serving-cert\") pod \"console-5f6988b95b-7w4kf\" (UID: \"8d33609e-12bd-4b5f-ba47-7bb64348eb3e\") " pod="openshift-console/console-5f6988b95b-7w4kf" Jul 06 18:26:22.514292 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:22.514190 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvt4b\" (UniqueName: \"kubernetes.io/projected/8d33609e-12bd-4b5f-ba47-7bb64348eb3e-kube-api-access-lvt4b\") pod \"console-5f6988b95b-7w4kf\" (UID: \"8d33609e-12bd-4b5f-ba47-7bb64348eb3e\") " pod="openshift-console/console-5f6988b95b-7w4kf" Jul 06 18:26:22.514292 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:22.514222 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8d33609e-12bd-4b5f-ba47-7bb64348eb3e-console-serving-cert\") pod \"console-5f6988b95b-7w4kf\" (UID: \"8d33609e-12bd-4b5f-ba47-7bb64348eb3e\") " pod="openshift-console/console-5f6988b95b-7w4kf" Jul 06 18:26:22.514490 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:22.514308 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8d33609e-12bd-4b5f-ba47-7bb64348eb3e-console-config\") pod \"console-5f6988b95b-7w4kf\" (UID: \"8d33609e-12bd-4b5f-ba47-7bb64348eb3e\") " pod="openshift-console/console-5f6988b95b-7w4kf" Jul 06 18:26:22.514490 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:22.514350 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8d33609e-12bd-4b5f-ba47-7bb64348eb3e-service-ca\") pod \"console-5f6988b95b-7w4kf\" (UID: \"8d33609e-12bd-4b5f-ba47-7bb64348eb3e\") " pod="openshift-console/console-5f6988b95b-7w4kf" Jul 06 18:26:22.597403 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:22.597326 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:22.615878 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:22.615817 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8d33609e-12bd-4b5f-ba47-7bb64348eb3e-console-oauth-config\") pod \"console-5f6988b95b-7w4kf\" (UID: \"8d33609e-12bd-4b5f-ba47-7bb64348eb3e\") " pod="openshift-console/console-5f6988b95b-7w4kf" Jul 06 18:26:22.615878 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:22.615882 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d33609e-12bd-4b5f-ba47-7bb64348eb3e-trusted-ca-bundle\") pod \"console-5f6988b95b-7w4kf\" (UID: \"8d33609e-12bd-4b5f-ba47-7bb64348eb3e\") " pod="openshift-console/console-5f6988b95b-7w4kf" Jul 06 18:26:22.616118 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:22.615916 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8d33609e-12bd-4b5f-ba47-7bb64348eb3e-oauth-serving-cert\") pod \"console-5f6988b95b-7w4kf\" (UID: \"8d33609e-12bd-4b5f-ba47-7bb64348eb3e\") " pod="openshift-console/console-5f6988b95b-7w4kf" Jul 06 18:26:22.616118 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:22.615933 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvt4b\" (UniqueName: \"kubernetes.io/projected/8d33609e-12bd-4b5f-ba47-7bb64348eb3e-kube-api-access-lvt4b\") pod \"console-5f6988b95b-7w4kf\" (UID: \"8d33609e-12bd-4b5f-ba47-7bb64348eb3e\") " pod="openshift-console/console-5f6988b95b-7w4kf" Jul 06 18:26:22.616118 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:22.615966 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8d33609e-12bd-4b5f-ba47-7bb64348eb3e-console-serving-cert\") pod \"console-5f6988b95b-7w4kf\" (UID: \"8d33609e-12bd-4b5f-ba47-7bb64348eb3e\") " pod="openshift-console/console-5f6988b95b-7w4kf" Jul 06 18:26:22.616118 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:22.616040 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8d33609e-12bd-4b5f-ba47-7bb64348eb3e-console-config\") pod \"console-5f6988b95b-7w4kf\" (UID: \"8d33609e-12bd-4b5f-ba47-7bb64348eb3e\") " pod="openshift-console/console-5f6988b95b-7w4kf" Jul 06 18:26:22.616118 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:22.616077 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8d33609e-12bd-4b5f-ba47-7bb64348eb3e-service-ca\") pod \"console-5f6988b95b-7w4kf\" (UID: \"8d33609e-12bd-4b5f-ba47-7bb64348eb3e\") " pod="openshift-console/console-5f6988b95b-7w4kf" Jul 06 18:26:22.616761 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:22.616715 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8d33609e-12bd-4b5f-ba47-7bb64348eb3e-oauth-serving-cert\") pod \"console-5f6988b95b-7w4kf\" (UID: \"8d33609e-12bd-4b5f-ba47-7bb64348eb3e\") " pod="openshift-console/console-5f6988b95b-7w4kf" Jul 06 18:26:22.616924 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:22.616763 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8d33609e-12bd-4b5f-ba47-7bb64348eb3e-service-ca\") pod \"console-5f6988b95b-7w4kf\" (UID: \"8d33609e-12bd-4b5f-ba47-7bb64348eb3e\") " pod="openshift-console/console-5f6988b95b-7w4kf" Jul 06 18:26:22.617123 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:22.617079 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d33609e-12bd-4b5f-ba47-7bb64348eb3e-trusted-ca-bundle\") pod \"console-5f6988b95b-7w4kf\" (UID: \"8d33609e-12bd-4b5f-ba47-7bb64348eb3e\") " pod="openshift-console/console-5f6988b95b-7w4kf" Jul 06 18:26:22.617277 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:22.617251 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8d33609e-12bd-4b5f-ba47-7bb64348eb3e-console-config\") pod \"console-5f6988b95b-7w4kf\" (UID: \"8d33609e-12bd-4b5f-ba47-7bb64348eb3e\") " pod="openshift-console/console-5f6988b95b-7w4kf" Jul 06 18:26:22.619092 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:22.619070 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8d33609e-12bd-4b5f-ba47-7bb64348eb3e-console-serving-cert\") pod \"console-5f6988b95b-7w4kf\" (UID: \"8d33609e-12bd-4b5f-ba47-7bb64348eb3e\") " pod="openshift-console/console-5f6988b95b-7w4kf" Jul 06 18:26:22.619191 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:22.619087 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8d33609e-12bd-4b5f-ba47-7bb64348eb3e-console-oauth-config\") pod \"console-5f6988b95b-7w4kf\" (UID: \"8d33609e-12bd-4b5f-ba47-7bb64348eb3e\") " pod="openshift-console/console-5f6988b95b-7w4kf" Jul 06 18:26:22.629952 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:22.629923 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvt4b\" (UniqueName: \"kubernetes.io/projected/8d33609e-12bd-4b5f-ba47-7bb64348eb3e-kube-api-access-lvt4b\") pod \"console-5f6988b95b-7w4kf\" (UID: \"8d33609e-12bd-4b5f-ba47-7bb64348eb3e\") " pod="openshift-console/console-5f6988b95b-7w4kf" Jul 06 18:26:22.691247 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:22.691210 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:22.790691 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:22.790643 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f6988b95b-7w4kf" Jul 06 18:26:22.796316 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:22.796285 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:23.277787 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:23.277751 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:23.377782 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:23.377754 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:23.603381 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:23.603189 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f6988b95b-7w4kf"] Jul 06 18:26:23.610896 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:26:23.610835 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d33609e_12bd_4b5f_ba47_7bb64348eb3e.slice/crio-5babae7973e0ab2b1c68bedafd0fb4147623c04cce50a9e9f36f6aa10e8d6148 WatchSource:0}: Error finding container 5babae7973e0ab2b1c68bedafd0fb4147623c04cce50a9e9f36f6aa10e8d6148: Status 404 returned error can't find the container with id 5babae7973e0ab2b1c68bedafd0fb4147623c04cce50a9e9f36f6aa10e8d6148 Jul 06 18:26:23.892961 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:23.892929 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gtbr8" event={"ID":"0c313f99-6b41-44dc-9180-f5e686dd4e69","Type":"ContainerStarted","Data":"dbcce499854cd10fcabf446d4a34a230d090062fdb4347708d288773bb316cf3"} Jul 06 18:26:23.893141 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:23.893045 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gtbr8" Jul 06 18:26:23.895311 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:23.895282 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qmdl6" event={"ID":"66bc78d7-1b98-4795-9097-641e96f71ba1","Type":"ContainerStarted","Data":"b9c030c293aafa5977e09623cd2b3ecfb42c5b428652accf633fd64582239dd4"} Jul 06 18:26:23.895445 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:23.895384 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qmdl6" Jul 06 18:26:23.897444 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:23.897416 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f6988b95b-7w4kf" event={"ID":"8d33609e-12bd-4b5f-ba47-7bb64348eb3e","Type":"ContainerStarted","Data":"334d95cfac9c80828014fd0e3fa8fae263c8d94d22cab2a892b4e9af47a28a19"} Jul 06 18:26:23.897542 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:23.897461 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f6988b95b-7w4kf" event={"ID":"8d33609e-12bd-4b5f-ba47-7bb64348eb3e","Type":"ContainerStarted","Data":"5babae7973e0ab2b1c68bedafd0fb4147623c04cce50a9e9f36f6aa10e8d6148"} Jul 06 18:26:23.912588 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:23.912508 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gtbr8" podStartSLOduration=1.7238255329999999 podStartE2EDuration="7.912490368s" totalImagesPullingTime="6.188664835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:26:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:26:23.911340349 +0000 UTC m=+400.637769345" watchObservedRunningTime="2026-07-06 18:26:23.912490368 +0000 UTC m=+400.638919328" Jul 06 18:26:23.929130 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:23.929082 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qmdl6" podStartSLOduration=1.492566394 podStartE2EDuration="4.929065889s" totalImagesPullingTime="3.436499495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:26:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:26:23.927859483 +0000 UTC m=+400.654288442" watchObservedRunningTime="2026-07-06 18:26:23.929065889 +0000 UTC m=+400.655494847" Jul 06 18:26:23.948768 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:23.948710 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f6988b95b-7w4kf" podStartSLOduration=1.94869593 podStartE2EDuration="1.94869593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:26:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:26:23.946500316 +0000 UTC m=+400.672929288" watchObservedRunningTime="2026-07-06 18:26:23.94869593 +0000 UTC m=+400.675124888" Jul 06 18:26:24.133792 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:24.133755 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:25.887464 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:25.887414 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:25.988917 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:25.988878 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:26.246924 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:26.246897 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:26.693304 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:26.693275 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:32.791234 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:32.791185 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f6988b95b-7w4kf" Jul 06 18:26:32.791653 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:32.791268 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f6988b95b-7w4kf" Jul 06 18:26:32.796773 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:32.796747 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f6988b95b-7w4kf" Jul 06 18:26:32.957437 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:32.957407 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f6988b95b-7w4kf" Jul 06 18:26:33.028188 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:33.028155 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58b479586d-rf8wx"] Jul 06 18:26:33.146046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:33.146008 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:33.243600 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:33.243567 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:33.345794 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:33.345753 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:34.913113 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:34.913070 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gtbr8" Jul 06 18:26:34.913510 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:34.913255 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qmdl6" Jul 06 18:26:35.126007 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:35.125968 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:35.224780 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:35.224704 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:36.688710 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:36.688675 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gtbr8"] Jul 06 18:26:36.689192 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:36.688931 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gtbr8" podUID="0c313f99-6b41-44dc-9180-f5e686dd4e69" containerName="manager" containerID="cri-o://dbcce499854cd10fcabf446d4a34a230d090062fdb4347708d288773bb316cf3" gracePeriod=2 Jul 06 18:26:36.694272 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:26:36.694232 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-gtbr8\" is forbidden: User \"system:node:ip-10-0-128-98.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-98.ec2.internal' and this object" podUID="0c313f99-6b41-44dc-9180-f5e686dd4e69" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gtbr8" Jul 06 18:26:36.696482 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:36.696457 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gtbr8"] Jul 06 18:26:36.727782 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:36.727746 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6jpb"] Jul 06 18:26:36.728148 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:36.728133 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c313f99-6b41-44dc-9180-f5e686dd4e69" containerName="manager" Jul 06 18:26:36.728148 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:36.728148 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c313f99-6b41-44dc-9180-f5e686dd4e69" containerName="manager" Jul 06 18:26:36.728308 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:36.728232 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c313f99-6b41-44dc-9180-f5e686dd4e69" containerName="manager" Jul 06 18:26:36.731512 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:36.731488 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6jpb" Jul 06 18:26:36.734209 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:26:36.734173 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-gtbr8\" is forbidden: User \"system:node:ip-10-0-128-98.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-98.ec2.internal' and this object" podUID="0c313f99-6b41-44dc-9180-f5e686dd4e69" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gtbr8" Jul 06 18:26:36.762850 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:36.762811 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6jpb"] Jul 06 18:26:36.850244 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:36.850208 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz99j\" (UniqueName: \"kubernetes.io/projected/d468d803-d992-4227-9b8d-e763ebc36740-kube-api-access-lz99j\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n6jpb\" (UID: \"d468d803-d992-4227-9b8d-e763ebc36740\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6jpb" Jul 06 18:26:36.850407 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:36.850281 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d468d803-d992-4227-9b8d-e763ebc36740-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n6jpb\" (UID: \"d468d803-d992-4227-9b8d-e763ebc36740\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6jpb" Jul 06 18:26:36.934728 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:36.934702 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gtbr8" Jul 06 18:26:36.938351 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:26:36.938321 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-gtbr8\" is forbidden: User \"system:node:ip-10-0-128-98.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-98.ec2.internal' and this object" podUID="0c313f99-6b41-44dc-9180-f5e686dd4e69" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gtbr8" Jul 06 18:26:36.951790 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:36.951727 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d468d803-d992-4227-9b8d-e763ebc36740-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n6jpb\" (UID: \"d468d803-d992-4227-9b8d-e763ebc36740\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6jpb" Jul 06 18:26:36.951933 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:36.951821 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz99j\" (UniqueName: \"kubernetes.io/projected/d468d803-d992-4227-9b8d-e763ebc36740-kube-api-access-lz99j\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n6jpb\" (UID: \"d468d803-d992-4227-9b8d-e763ebc36740\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6jpb" Jul 06 18:26:36.952131 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:36.952111 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d468d803-d992-4227-9b8d-e763ebc36740-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n6jpb\" (UID: \"d468d803-d992-4227-9b8d-e763ebc36740\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6jpb" Jul 06 18:26:36.960827 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:36.960802 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz99j\" (UniqueName: \"kubernetes.io/projected/d468d803-d992-4227-9b8d-e763ebc36740-kube-api-access-lz99j\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n6jpb\" (UID: \"d468d803-d992-4227-9b8d-e763ebc36740\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6jpb" Jul 06 18:26:36.975443 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:36.975419 2566 generic.go:356] "Generic (PLEG): container finished" podID="0c313f99-6b41-44dc-9180-f5e686dd4e69" containerID="dbcce499854cd10fcabf446d4a34a230d090062fdb4347708d288773bb316cf3" exitCode=0 Jul 06 18:26:36.975552 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:36.975467 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gtbr8" Jul 06 18:26:36.975552 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:36.975520 2566 scope.go:117] "RemoveContainer" containerID="dbcce499854cd10fcabf446d4a34a230d090062fdb4347708d288773bb316cf3" Jul 06 18:26:36.978124 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:26:36.978094 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-gtbr8\" is forbidden: User \"system:node:ip-10-0-128-98.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-98.ec2.internal' and this object" podUID="0c313f99-6b41-44dc-9180-f5e686dd4e69" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gtbr8" Jul 06 18:26:36.984139 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:36.984115 2566 scope.go:117] "RemoveContainer" containerID="dbcce499854cd10fcabf446d4a34a230d090062fdb4347708d288773bb316cf3" Jul 06 18:26:36.984385 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:26:36.984364 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbcce499854cd10fcabf446d4a34a230d090062fdb4347708d288773bb316cf3\": container with ID starting with dbcce499854cd10fcabf446d4a34a230d090062fdb4347708d288773bb316cf3 not found: ID does not exist" containerID="dbcce499854cd10fcabf446d4a34a230d090062fdb4347708d288773bb316cf3" Jul 06 18:26:36.984441 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:36.984396 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbcce499854cd10fcabf446d4a34a230d090062fdb4347708d288773bb316cf3"} err="failed to get container status \"dbcce499854cd10fcabf446d4a34a230d090062fdb4347708d288773bb316cf3\": rpc error: code = NotFound desc = could not find container \"dbcce499854cd10fcabf446d4a34a230d090062fdb4347708d288773bb316cf3\": container with ID starting with dbcce499854cd10fcabf446d4a34a230d090062fdb4347708d288773bb316cf3 not found: ID does not exist" Jul 06 18:26:37.052955 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:37.052926 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c313f99-6b41-44dc-9180-f5e686dd4e69-extensions-socket-volume\") pod \"0c313f99-6b41-44dc-9180-f5e686dd4e69\" (UID: \"0c313f99-6b41-44dc-9180-f5e686dd4e69\") " Jul 06 18:26:37.053103 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:37.053001 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rwhj\" (UniqueName: \"kubernetes.io/projected/0c313f99-6b41-44dc-9180-f5e686dd4e69-kube-api-access-9rwhj\") pod \"0c313f99-6b41-44dc-9180-f5e686dd4e69\" (UID: \"0c313f99-6b41-44dc-9180-f5e686dd4e69\") " Jul 06 18:26:37.053511 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:37.053486 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c313f99-6b41-44dc-9180-f5e686dd4e69-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0c313f99-6b41-44dc-9180-f5e686dd4e69" (UID: "0c313f99-6b41-44dc-9180-f5e686dd4e69"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:26:37.055248 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:37.055228 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c313f99-6b41-44dc-9180-f5e686dd4e69-kube-api-access-9rwhj" (OuterVolumeSpecName: "kube-api-access-9rwhj") pod "0c313f99-6b41-44dc-9180-f5e686dd4e69" (UID: "0c313f99-6b41-44dc-9180-f5e686dd4e69"). InnerVolumeSpecName "kube-api-access-9rwhj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:26:37.068493 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:37.068463 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6jpb" Jul 06 18:26:37.153940 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:37.153906 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rwhj\" (UniqueName: \"kubernetes.io/projected/0c313f99-6b41-44dc-9180-f5e686dd4e69-kube-api-access-9rwhj\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:26:37.153940 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:37.153934 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c313f99-6b41-44dc-9180-f5e686dd4e69-extensions-socket-volume\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:26:37.217086 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:37.217059 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6jpb"] Jul 06 18:26:37.218674 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:26:37.218646 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd468d803_d992_4227_9b8d_e763ebc36740.slice/crio-a12dba9b949e38ea085b1b8bf33fee36bcc82592c59e59796e88657c59c6a29f WatchSource:0}: Error finding container a12dba9b949e38ea085b1b8bf33fee36bcc82592c59e59796e88657c59c6a29f: Status 404 returned error can't find the container with id a12dba9b949e38ea085b1b8bf33fee36bcc82592c59e59796e88657c59c6a29f Jul 06 18:26:37.287308 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:26:37.287276 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-gtbr8\" is forbidden: User \"system:node:ip-10-0-128-98.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-98.ec2.internal' and this object" podUID="0c313f99-6b41-44dc-9180-f5e686dd4e69" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gtbr8" Jul 06 18:26:37.705208 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:37.705174 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:37.826058 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:37.826026 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:37.849031 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:37.848991 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c313f99-6b41-44dc-9180-f5e686dd4e69" path="/var/lib/kubelet/pods/0c313f99-6b41-44dc-9180-f5e686dd4e69/volumes" Jul 06 18:26:37.982968 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:37.982848 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6jpb" event={"ID":"d468d803-d992-4227-9b8d-e763ebc36740","Type":"ContainerStarted","Data":"fdcb6cbf8ec533d72cc702781eb36281e7c87808209ade60f5105d4ed1813929"} Jul 06 18:26:37.982968 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:37.982937 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6jpb" event={"ID":"d468d803-d992-4227-9b8d-e763ebc36740","Type":"ContainerStarted","Data":"a12dba9b949e38ea085b1b8bf33fee36bcc82592c59e59796e88657c59c6a29f"} Jul 06 18:26:37.982968 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:37.982969 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6jpb" Jul 06 18:26:38.012842 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:38.012789 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6jpb" podStartSLOduration=2.012771584 podStartE2EDuration="2.012771584s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:26:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:26:38.010665231 +0000 UTC m=+414.737094190" watchObservedRunningTime="2026-07-06 18:26:38.012771584 +0000 UTC m=+414.739200542" Jul 06 18:26:42.318916 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:42.318861 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:42.418961 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:42.418930 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:48.096007 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:48.095970 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:48.208481 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:48.208440 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:48.992558 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:48.992527 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6jpb" Jul 06 18:26:54.296536 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:54.296499 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6jpb"] Jul 06 18:26:54.296976 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:54.296703 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6jpb" podUID="d468d803-d992-4227-9b8d-e763ebc36740" containerName="manager" containerID="cri-o://fdcb6cbf8ec533d72cc702781eb36281e7c87808209ade60f5105d4ed1813929" gracePeriod=10 Jul 06 18:26:54.557117 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:54.557045 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6jpb" Jul 06 18:26:54.631199 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:54.631163 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d468d803-d992-4227-9b8d-e763ebc36740-extensions-socket-volume\") pod \"d468d803-d992-4227-9b8d-e763ebc36740\" (UID: \"d468d803-d992-4227-9b8d-e763ebc36740\") " Jul 06 18:26:54.631199 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:54.631203 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lz99j\" (UniqueName: \"kubernetes.io/projected/d468d803-d992-4227-9b8d-e763ebc36740-kube-api-access-lz99j\") pod \"d468d803-d992-4227-9b8d-e763ebc36740\" (UID: \"d468d803-d992-4227-9b8d-e763ebc36740\") " Jul 06 18:26:54.631593 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:54.631569 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d468d803-d992-4227-9b8d-e763ebc36740-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d468d803-d992-4227-9b8d-e763ebc36740" (UID: "d468d803-d992-4227-9b8d-e763ebc36740"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:26:54.633379 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:54.633353 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d468d803-d992-4227-9b8d-e763ebc36740-kube-api-access-lz99j" (OuterVolumeSpecName: "kube-api-access-lz99j") pod "d468d803-d992-4227-9b8d-e763ebc36740" (UID: "d468d803-d992-4227-9b8d-e763ebc36740"). InnerVolumeSpecName "kube-api-access-lz99j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:26:54.732211 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:54.732176 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d468d803-d992-4227-9b8d-e763ebc36740-extensions-socket-volume\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:26:54.732211 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:54.732205 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lz99j\" (UniqueName: \"kubernetes.io/projected/d468d803-d992-4227-9b8d-e763ebc36740-kube-api-access-lz99j\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:26:55.077173 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:55.077136 2566 generic.go:356] "Generic (PLEG): container finished" podID="d468d803-d992-4227-9b8d-e763ebc36740" containerID="fdcb6cbf8ec533d72cc702781eb36281e7c87808209ade60f5105d4ed1813929" exitCode=0 Jul 06 18:26:55.077357 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:55.077184 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6jpb" Jul 06 18:26:55.077357 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:55.077223 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6jpb" event={"ID":"d468d803-d992-4227-9b8d-e763ebc36740","Type":"ContainerDied","Data":"fdcb6cbf8ec533d72cc702781eb36281e7c87808209ade60f5105d4ed1813929"} Jul 06 18:26:55.077357 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:55.077279 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6jpb" event={"ID":"d468d803-d992-4227-9b8d-e763ebc36740","Type":"ContainerDied","Data":"a12dba9b949e38ea085b1b8bf33fee36bcc82592c59e59796e88657c59c6a29f"} Jul 06 18:26:55.077357 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:55.077302 2566 scope.go:117] "RemoveContainer" containerID="fdcb6cbf8ec533d72cc702781eb36281e7c87808209ade60f5105d4ed1813929" Jul 06 18:26:55.086863 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:55.086845 2566 scope.go:117] "RemoveContainer" containerID="fdcb6cbf8ec533d72cc702781eb36281e7c87808209ade60f5105d4ed1813929" Jul 06 18:26:55.087142 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:26:55.087119 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdcb6cbf8ec533d72cc702781eb36281e7c87808209ade60f5105d4ed1813929\": container with ID starting with fdcb6cbf8ec533d72cc702781eb36281e7c87808209ade60f5105d4ed1813929 not found: ID does not exist" containerID="fdcb6cbf8ec533d72cc702781eb36281e7c87808209ade60f5105d4ed1813929" Jul 06 18:26:55.087236 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:55.087146 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdcb6cbf8ec533d72cc702781eb36281e7c87808209ade60f5105d4ed1813929"} err="failed to get container status \"fdcb6cbf8ec533d72cc702781eb36281e7c87808209ade60f5105d4ed1813929\": rpc error: code = NotFound desc = could not find container \"fdcb6cbf8ec533d72cc702781eb36281e7c87808209ade60f5105d4ed1813929\": container with ID starting with fdcb6cbf8ec533d72cc702781eb36281e7c87808209ade60f5105d4ed1813929 not found: ID does not exist" Jul 06 18:26:55.108290 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:55.108259 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6jpb"] Jul 06 18:26:55.110375 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:55.110356 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n6jpb"] Jul 06 18:26:55.849379 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:55.849345 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d468d803-d992-4227-9b8d-e763ebc36740" path="/var/lib/kubelet/pods/d468d803-d992-4227-9b8d-e763ebc36740/volumes" Jul 06 18:26:57.666150 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:57.666113 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:57.728961 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:57.728929 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:58.058049 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:58.057962 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-58b479586d-rf8wx" podUID="7ad4e548-2868-43ec-b0fc-dae0d329ab18" containerName="console" containerID="cri-o://ad2d8a83e03c42af245dbb178181e50a247bb2975ea2e14d919295660aa65b45" gracePeriod=15 Jul 06 18:26:58.299243 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:58.299207 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:58.304518 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:58.304494 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58b479586d-rf8wx" Jul 06 18:26:58.362809 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:58.362691 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7ad4e548-2868-43ec-b0fc-dae0d329ab18-console-oauth-config\") pod \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\" (UID: \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\") " Jul 06 18:26:58.362809 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:58.362779 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ad4e548-2868-43ec-b0fc-dae0d329ab18-trusted-ca-bundle\") pod \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\" (UID: \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\") " Jul 06 18:26:58.363077 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:58.362814 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7ad4e548-2868-43ec-b0fc-dae0d329ab18-oauth-serving-cert\") pod \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\" (UID: \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\") " Jul 06 18:26:58.363077 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:58.362904 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ss7m\" (UniqueName: \"kubernetes.io/projected/7ad4e548-2868-43ec-b0fc-dae0d329ab18-kube-api-access-9ss7m\") pod \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\" (UID: \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\") " Jul 06 18:26:58.363077 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:58.362941 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7ad4e548-2868-43ec-b0fc-dae0d329ab18-service-ca\") pod \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\" (UID: \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\") " Jul 06 18:26:58.363077 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:58.362963 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7ad4e548-2868-43ec-b0fc-dae0d329ab18-console-config\") pod \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\" (UID: \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\") " Jul 06 18:26:58.363077 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:58.363025 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7ad4e548-2868-43ec-b0fc-dae0d329ab18-console-serving-cert\") pod \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\" (UID: \"7ad4e548-2868-43ec-b0fc-dae0d329ab18\") " Jul 06 18:26:58.363557 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:58.363526 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ad4e548-2868-43ec-b0fc-dae0d329ab18-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7ad4e548-2868-43ec-b0fc-dae0d329ab18" (UID: "7ad4e548-2868-43ec-b0fc-dae0d329ab18"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:26:58.364001 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:58.363977 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ad4e548-2868-43ec-b0fc-dae0d329ab18-console-config" (OuterVolumeSpecName: "console-config") pod "7ad4e548-2868-43ec-b0fc-dae0d329ab18" (UID: "7ad4e548-2868-43ec-b0fc-dae0d329ab18"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:26:58.364256 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:58.364200 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ad4e548-2868-43ec-b0fc-dae0d329ab18-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7ad4e548-2868-43ec-b0fc-dae0d329ab18" (UID: "7ad4e548-2868-43ec-b0fc-dae0d329ab18"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:26:58.364360 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:58.364265 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ad4e548-2868-43ec-b0fc-dae0d329ab18-service-ca" (OuterVolumeSpecName: "service-ca") pod "7ad4e548-2868-43ec-b0fc-dae0d329ab18" (UID: "7ad4e548-2868-43ec-b0fc-dae0d329ab18"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:26:58.366087 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:58.365780 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ad4e548-2868-43ec-b0fc-dae0d329ab18-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7ad4e548-2868-43ec-b0fc-dae0d329ab18" (UID: "7ad4e548-2868-43ec-b0fc-dae0d329ab18"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:26:58.366220 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:58.366197 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ad4e548-2868-43ec-b0fc-dae0d329ab18-kube-api-access-9ss7m" (OuterVolumeSpecName: "kube-api-access-9ss7m") pod "7ad4e548-2868-43ec-b0fc-dae0d329ab18" (UID: "7ad4e548-2868-43ec-b0fc-dae0d329ab18"). InnerVolumeSpecName "kube-api-access-9ss7m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:26:58.366220 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:58.366204 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ad4e548-2868-43ec-b0fc-dae0d329ab18-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7ad4e548-2868-43ec-b0fc-dae0d329ab18" (UID: "7ad4e548-2868-43ec-b0fc-dae0d329ab18"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:26:58.412568 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:58.412531 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:26:58.463953 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:58.463920 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7ad4e548-2868-43ec-b0fc-dae0d329ab18-oauth-serving-cert\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:26:58.463953 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:58.463947 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ss7m\" (UniqueName: \"kubernetes.io/projected/7ad4e548-2868-43ec-b0fc-dae0d329ab18-kube-api-access-9ss7m\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:26:58.464149 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:58.463961 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7ad4e548-2868-43ec-b0fc-dae0d329ab18-service-ca\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:26:58.464149 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:58.463971 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7ad4e548-2868-43ec-b0fc-dae0d329ab18-console-config\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:26:58.464149 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:58.463979 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7ad4e548-2868-43ec-b0fc-dae0d329ab18-console-serving-cert\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:26:58.464149 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:58.463987 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7ad4e548-2868-43ec-b0fc-dae0d329ab18-console-oauth-config\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:26:58.464149 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:58.463995 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ad4e548-2868-43ec-b0fc-dae0d329ab18-trusted-ca-bundle\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:26:59.101565 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:59.101529 2566 generic.go:356] "Generic (PLEG): container finished" podID="7ad4e548-2868-43ec-b0fc-dae0d329ab18" containerID="ad2d8a83e03c42af245dbb178181e50a247bb2975ea2e14d919295660aa65b45" exitCode=0 Jul 06 18:26:59.102081 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:59.101572 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58b479586d-rf8wx" Jul 06 18:26:59.102081 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:59.101625 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58b479586d-rf8wx" event={"ID":"7ad4e548-2868-43ec-b0fc-dae0d329ab18","Type":"ContainerDied","Data":"ad2d8a83e03c42af245dbb178181e50a247bb2975ea2e14d919295660aa65b45"} Jul 06 18:26:59.102081 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:59.101683 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58b479586d-rf8wx" event={"ID":"7ad4e548-2868-43ec-b0fc-dae0d329ab18","Type":"ContainerDied","Data":"6e087ed58ae7ee22111b4c60d8109f82990339edfef5688cbeb43128fa08dfc0"} Jul 06 18:26:59.102081 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:59.101715 2566 scope.go:117] "RemoveContainer" containerID="ad2d8a83e03c42af245dbb178181e50a247bb2975ea2e14d919295660aa65b45" Jul 06 18:26:59.111166 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:59.111148 2566 scope.go:117] "RemoveContainer" containerID="ad2d8a83e03c42af245dbb178181e50a247bb2975ea2e14d919295660aa65b45" Jul 06 18:26:59.111434 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:26:59.111413 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad2d8a83e03c42af245dbb178181e50a247bb2975ea2e14d919295660aa65b45\": container with ID starting with ad2d8a83e03c42af245dbb178181e50a247bb2975ea2e14d919295660aa65b45 not found: ID does not exist" containerID="ad2d8a83e03c42af245dbb178181e50a247bb2975ea2e14d919295660aa65b45" Jul 06 18:26:59.111492 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:59.111436 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad2d8a83e03c42af245dbb178181e50a247bb2975ea2e14d919295660aa65b45"} err="failed to get container status \"ad2d8a83e03c42af245dbb178181e50a247bb2975ea2e14d919295660aa65b45\": rpc error: code = NotFound desc = could not find container \"ad2d8a83e03c42af245dbb178181e50a247bb2975ea2e14d919295660aa65b45\": container with ID starting with ad2d8a83e03c42af245dbb178181e50a247bb2975ea2e14d919295660aa65b45 not found: ID does not exist" Jul 06 18:26:59.124203 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:59.124180 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58b479586d-rf8wx"] Jul 06 18:26:59.128480 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:59.128460 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-58b479586d-rf8wx"] Jul 06 18:26:59.849106 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:26:59.849072 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ad4e548-2868-43ec-b0fc-dae0d329ab18" path="/var/lib/kubelet/pods/7ad4e548-2868-43ec-b0fc-dae0d329ab18/volumes" Jul 06 18:27:01.871254 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:01.871218 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:27:01.972195 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:01.972162 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:27:05.470644 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:05.470560 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:27:05.571032 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:05.571000 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:27:09.875505 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:09.875467 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:27:09.974564 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:09.974534 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:27:10.883931 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:10.883895 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4"] Jul 06 18:27:10.884406 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:10.884261 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ad4e548-2868-43ec-b0fc-dae0d329ab18" containerName="console" Jul 06 18:27:10.884406 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:10.884272 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ad4e548-2868-43ec-b0fc-dae0d329ab18" containerName="console" Jul 06 18:27:10.884406 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:10.884284 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d468d803-d992-4227-9b8d-e763ebc36740" containerName="manager" Jul 06 18:27:10.884406 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:10.884289 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d468d803-d992-4227-9b8d-e763ebc36740" containerName="manager" Jul 06 18:27:10.884406 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:10.884344 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d468d803-d992-4227-9b8d-e763ebc36740" containerName="manager" Jul 06 18:27:10.884406 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:10.884352 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ad4e548-2868-43ec-b0fc-dae0d329ab18" containerName="console" Jul 06 18:27:10.886511 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:10.886490 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:10.889484 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:10.889461 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-94l7n\"" Jul 06 18:27:10.903548 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:10.903517 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4"] Jul 06 18:27:10.924901 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:10.924849 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:27:10.978659 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:10.978626 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82159f17-c966-4747-aec4-1dd75e8726d5-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-p5nb4\" (UID: \"82159f17-c966-4747-aec4-1dd75e8726d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:10.978851 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:10.978665 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82159f17-c966-4747-aec4-1dd75e8726d5-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-p5nb4\" (UID: \"82159f17-c966-4747-aec4-1dd75e8726d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:10.978851 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:10.978686 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82159f17-c966-4747-aec4-1dd75e8726d5-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-p5nb4\" (UID: \"82159f17-c966-4747-aec4-1dd75e8726d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:10.978851 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:10.978780 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82159f17-c966-4747-aec4-1dd75e8726d5-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-p5nb4\" (UID: \"82159f17-c966-4747-aec4-1dd75e8726d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:10.979071 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:10.978854 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82159f17-c966-4747-aec4-1dd75e8726d5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-p5nb4\" (UID: \"82159f17-c966-4747-aec4-1dd75e8726d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:10.979071 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:10.978948 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82159f17-c966-4747-aec4-1dd75e8726d5-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-p5nb4\" (UID: \"82159f17-c966-4747-aec4-1dd75e8726d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:10.979071 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:10.978985 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82159f17-c966-4747-aec4-1dd75e8726d5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-p5nb4\" (UID: \"82159f17-c966-4747-aec4-1dd75e8726d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:10.979071 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:10.979014 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7xwf\" (UniqueName: \"kubernetes.io/projected/82159f17-c966-4747-aec4-1dd75e8726d5-kube-api-access-m7xwf\") pod \"maas-default-gateway-openshift-default-56654dcf7-p5nb4\" (UID: \"82159f17-c966-4747-aec4-1dd75e8726d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:10.979071 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:10.979065 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82159f17-c966-4747-aec4-1dd75e8726d5-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-p5nb4\" (UID: \"82159f17-c966-4747-aec4-1dd75e8726d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:11.080014 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:11.079978 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82159f17-c966-4747-aec4-1dd75e8726d5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-p5nb4\" (UID: \"82159f17-c966-4747-aec4-1dd75e8726d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:11.080014 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:11.080020 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82159f17-c966-4747-aec4-1dd75e8726d5-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-p5nb4\" (UID: \"82159f17-c966-4747-aec4-1dd75e8726d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:11.080253 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:11.080039 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82159f17-c966-4747-aec4-1dd75e8726d5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-p5nb4\" (UID: \"82159f17-c966-4747-aec4-1dd75e8726d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:11.080253 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:11.080061 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7xwf\" (UniqueName: \"kubernetes.io/projected/82159f17-c966-4747-aec4-1dd75e8726d5-kube-api-access-m7xwf\") pod \"maas-default-gateway-openshift-default-56654dcf7-p5nb4\" (UID: \"82159f17-c966-4747-aec4-1dd75e8726d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:11.080253 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:11.080089 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82159f17-c966-4747-aec4-1dd75e8726d5-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-p5nb4\" (UID: \"82159f17-c966-4747-aec4-1dd75e8726d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:11.080253 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:11.080117 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82159f17-c966-4747-aec4-1dd75e8726d5-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-p5nb4\" (UID: \"82159f17-c966-4747-aec4-1dd75e8726d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:11.080253 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:11.080155 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82159f17-c966-4747-aec4-1dd75e8726d5-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-p5nb4\" (UID: \"82159f17-c966-4747-aec4-1dd75e8726d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:11.080253 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:11.080179 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82159f17-c966-4747-aec4-1dd75e8726d5-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-p5nb4\" (UID: \"82159f17-c966-4747-aec4-1dd75e8726d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:11.080253 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:11.080238 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82159f17-c966-4747-aec4-1dd75e8726d5-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-p5nb4\" (UID: \"82159f17-c966-4747-aec4-1dd75e8726d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:11.080609 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:11.080444 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82159f17-c966-4747-aec4-1dd75e8726d5-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-p5nb4\" (UID: \"82159f17-c966-4747-aec4-1dd75e8726d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:11.080609 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:11.080577 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82159f17-c966-4747-aec4-1dd75e8726d5-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-p5nb4\" (UID: \"82159f17-c966-4747-aec4-1dd75e8726d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:11.080686 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:11.080625 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82159f17-c966-4747-aec4-1dd75e8726d5-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-p5nb4\" (UID: \"82159f17-c966-4747-aec4-1dd75e8726d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:11.080686 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:11.080629 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82159f17-c966-4747-aec4-1dd75e8726d5-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-p5nb4\" (UID: \"82159f17-c966-4747-aec4-1dd75e8726d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:11.080816 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:11.080797 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82159f17-c966-4747-aec4-1dd75e8726d5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-p5nb4\" (UID: \"82159f17-c966-4747-aec4-1dd75e8726d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:11.082713 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:11.082694 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82159f17-c966-4747-aec4-1dd75e8726d5-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-p5nb4\" (UID: \"82159f17-c966-4747-aec4-1dd75e8726d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:11.082926 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:11.082908 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82159f17-c966-4747-aec4-1dd75e8726d5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-p5nb4\" (UID: \"82159f17-c966-4747-aec4-1dd75e8726d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:11.090359 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:11.090333 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7xwf\" (UniqueName: \"kubernetes.io/projected/82159f17-c966-4747-aec4-1dd75e8726d5-kube-api-access-m7xwf\") pod \"maas-default-gateway-openshift-default-56654dcf7-p5nb4\" (UID: \"82159f17-c966-4747-aec4-1dd75e8726d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:11.090472 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:11.090404 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82159f17-c966-4747-aec4-1dd75e8726d5-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-p5nb4\" (UID: \"82159f17-c966-4747-aec4-1dd75e8726d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:11.146806 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:11.146775 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:27:11.197741 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:11.197708 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:11.339178 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:11.339148 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4"] Jul 06 18:27:11.339547 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:27:11.339522 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82159f17_c966_4747_aec4_1dd75e8726d5.slice/crio-1288f8cf831472303b959cbf633f40accceab0175f4aad54d84f66822a65cf93 WatchSource:0}: Error finding container 1288f8cf831472303b959cbf633f40accceab0175f4aad54d84f66822a65cf93: Status 404 returned error can't find the container with id 1288f8cf831472303b959cbf633f40accceab0175f4aad54d84f66822a65cf93 Jul 06 18:27:11.341605 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:11.341573 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:27:11.341701 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:11.341649 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:27:11.341701 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:11.341694 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:27:12.176672 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:12.176631 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" event={"ID":"82159f17-c966-4747-aec4-1dd75e8726d5","Type":"ContainerStarted","Data":"88afcb9a738646b80a832cecee909adfc047b277d99d642499c98c8d58d81fc1"} Jul 06 18:27:12.177116 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:12.176700 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" event={"ID":"82159f17-c966-4747-aec4-1dd75e8726d5","Type":"ContainerStarted","Data":"1288f8cf831472303b959cbf633f40accceab0175f4aad54d84f66822a65cf93"} Jul 06 18:27:12.198362 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:12.198301 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:12.200795 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:12.200739 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" podStartSLOduration=2.200719983 podStartE2EDuration="2.200719983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:27:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:27:12.197375049 +0000 UTC m=+448.923804010" watchObservedRunningTime="2026-07-06 18:27:12.200719983 +0000 UTC m=+448.927148942" Jul 06 18:27:12.203888 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:12.203846 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:13.182237 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:13.182210 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:13.183343 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:13.183326 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:27:13.275474 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:13.275444 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:27:13.280309 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:13.280279 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-dpmpr"] Jul 06 18:27:13.282764 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:13.282747 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-dpmpr" Jul 06 18:27:13.285586 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:13.285564 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 06 18:27:13.285680 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:13.285573 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jqz8q\"" Jul 06 18:27:13.292391 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:13.292366 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-dpmpr"] Jul 06 18:27:13.378911 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:13.378878 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-dpmpr"] Jul 06 18:27:13.386149 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:13.386122 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:27:13.399339 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:13.399313 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bc1d60cd-b84e-49a6-a181-e1ba602eac63-config-file\") pod \"limitador-limitador-64c94fd64f-dpmpr\" (UID: \"bc1d60cd-b84e-49a6-a181-e1ba602eac63\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-dpmpr" Jul 06 18:27:13.399464 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:13.399376 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmz2c\" (UniqueName: \"kubernetes.io/projected/bc1d60cd-b84e-49a6-a181-e1ba602eac63-kube-api-access-cmz2c\") pod \"limitador-limitador-64c94fd64f-dpmpr\" (UID: \"bc1d60cd-b84e-49a6-a181-e1ba602eac63\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-dpmpr" Jul 06 18:27:13.444458 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:13.444374 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:27:13.501033 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:13.500998 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmz2c\" (UniqueName: \"kubernetes.io/projected/bc1d60cd-b84e-49a6-a181-e1ba602eac63-kube-api-access-cmz2c\") pod \"limitador-limitador-64c94fd64f-dpmpr\" (UID: \"bc1d60cd-b84e-49a6-a181-e1ba602eac63\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-dpmpr" Jul 06 18:27:13.501200 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:13.501101 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bc1d60cd-b84e-49a6-a181-e1ba602eac63-config-file\") pod \"limitador-limitador-64c94fd64f-dpmpr\" (UID: \"bc1d60cd-b84e-49a6-a181-e1ba602eac63\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-dpmpr" Jul 06 18:27:13.501698 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:13.501681 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bc1d60cd-b84e-49a6-a181-e1ba602eac63-config-file\") pod \"limitador-limitador-64c94fd64f-dpmpr\" (UID: \"bc1d60cd-b84e-49a6-a181-e1ba602eac63\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-dpmpr" Jul 06 18:27:13.509696 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:13.509667 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmz2c\" (UniqueName: \"kubernetes.io/projected/bc1d60cd-b84e-49a6-a181-e1ba602eac63-kube-api-access-cmz2c\") pod \"limitador-limitador-64c94fd64f-dpmpr\" (UID: \"bc1d60cd-b84e-49a6-a181-e1ba602eac63\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-dpmpr" Jul 06 18:27:13.595035 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:13.594995 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-dpmpr" Jul 06 18:27:13.758313 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:13.758289 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-dpmpr"] Jul 06 18:27:13.759859 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:27:13.759834 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc1d60cd_b84e_49a6_a181_e1ba602eac63.slice/crio-9d81cb241b8cf02a7ef224b35246a6d4dabaec71cbe785bf0eca27e700a021a8 WatchSource:0}: Error finding container 9d81cb241b8cf02a7ef224b35246a6d4dabaec71cbe785bf0eca27e700a021a8: Status 404 returned error can't find the container with id 9d81cb241b8cf02a7ef224b35246a6d4dabaec71cbe785bf0eca27e700a021a8 Jul 06 18:27:14.184806 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:14.184774 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-clf9h"] Jul 06 18:27:14.188201 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:14.188177 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-clf9h" Jul 06 18:27:14.190314 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:14.190287 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-dpmpr" event={"ID":"bc1d60cd-b84e-49a6-a181-e1ba602eac63","Type":"ContainerStarted","Data":"9d81cb241b8cf02a7ef224b35246a6d4dabaec71cbe785bf0eca27e700a021a8"} Jul 06 18:27:14.191325 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:14.191305 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-5bwhk\"" Jul 06 18:27:14.198397 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:14.198373 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-clf9h"] Jul 06 18:27:14.212976 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:14.212947 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:27:14.288195 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:14.288151 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:27:14.308444 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:14.308409 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dr46\" (UniqueName: \"kubernetes.io/projected/58335c1e-7f13-4d27-9d16-3d9b7e2d2f0c-kube-api-access-2dr46\") pod \"authorino-58cd955675-clf9h\" (UID: \"58335c1e-7f13-4d27-9d16-3d9b7e2d2f0c\") " pod="kuadrant-system/authorino-58cd955675-clf9h" Jul 06 18:27:14.409563 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:14.409527 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dr46\" (UniqueName: \"kubernetes.io/projected/58335c1e-7f13-4d27-9d16-3d9b7e2d2f0c-kube-api-access-2dr46\") pod \"authorino-58cd955675-clf9h\" (UID: \"58335c1e-7f13-4d27-9d16-3d9b7e2d2f0c\") " pod="kuadrant-system/authorino-58cd955675-clf9h" Jul 06 18:27:14.420086 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:14.420053 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dr46\" (UniqueName: \"kubernetes.io/projected/58335c1e-7f13-4d27-9d16-3d9b7e2d2f0c-kube-api-access-2dr46\") pod \"authorino-58cd955675-clf9h\" (UID: \"58335c1e-7f13-4d27-9d16-3d9b7e2d2f0c\") " pod="kuadrant-system/authorino-58cd955675-clf9h" Jul 06 18:27:14.499859 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:14.499771 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-clf9h" Jul 06 18:27:14.702679 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:14.702624 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-clf9h"] Jul 06 18:27:15.201064 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:15.200998 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-clf9h" event={"ID":"58335c1e-7f13-4d27-9d16-3d9b7e2d2f0c","Type":"ContainerStarted","Data":"f84317a398adefda02417af23ed3f2611293dbf59f9bad8909a3cde1ae2a8bf9"} Jul 06 18:27:18.569663 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:18.569620 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-clf9h"] Jul 06 18:27:18.668722 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:18.668686 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:27:18.767838 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:18.767807 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:27:19.229722 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:19.229679 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-clf9h" event={"ID":"58335c1e-7f13-4d27-9d16-3d9b7e2d2f0c","Type":"ContainerStarted","Data":"1f29e76812606afe87a5b64a5446752b5f70058a2a1a0079cc1ec5e0d934cb7f"} Jul 06 18:27:19.229722 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:19.229710 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-clf9h" podUID="58335c1e-7f13-4d27-9d16-3d9b7e2d2f0c" containerName="authorino" containerID="cri-o://1f29e76812606afe87a5b64a5446752b5f70058a2a1a0079cc1ec5e0d934cb7f" gracePeriod=30 Jul 06 18:27:19.231292 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:19.231267 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-dpmpr" event={"ID":"bc1d60cd-b84e-49a6-a181-e1ba602eac63","Type":"ContainerStarted","Data":"c559af0ce9074364cfc52a55c0c3d00af2888cc91656a8868e64f2bb03f136c6"} Jul 06 18:27:19.231417 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:19.231395 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-dpmpr" Jul 06 18:27:19.251458 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:19.251395 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-clf9h" podStartSLOduration=1.420948365 podStartE2EDuration="5.251374652s" totalImagesPullingTime="3.830426287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:27:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:27:19.246267354 +0000 UTC m=+455.972696317" watchObservedRunningTime="2026-07-06 18:27:19.251374652 +0000 UTC m=+455.977803611" Jul 06 18:27:19.268349 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:19.268298 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-dpmpr" podStartSLOduration=1.492292823 podStartE2EDuration="6.268284432s" totalImagesPullingTime="4.775991609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:27:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:27:19.267244757 +0000 UTC m=+455.993673716" watchObservedRunningTime="2026-07-06 18:27:19.268284432 +0000 UTC m=+455.994713395" Jul 06 18:27:19.483808 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:19.483744 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-clf9h" Jul 06 18:27:19.658351 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:19.658313 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dr46\" (UniqueName: \"kubernetes.io/projected/58335c1e-7f13-4d27-9d16-3d9b7e2d2f0c-kube-api-access-2dr46\") pod \"58335c1e-7f13-4d27-9d16-3d9b7e2d2f0c\" (UID: \"58335c1e-7f13-4d27-9d16-3d9b7e2d2f0c\") " Jul 06 18:27:19.660527 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:19.660498 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58335c1e-7f13-4d27-9d16-3d9b7e2d2f0c-kube-api-access-2dr46" (OuterVolumeSpecName: "kube-api-access-2dr46") pod "58335c1e-7f13-4d27-9d16-3d9b7e2d2f0c" (UID: "58335c1e-7f13-4d27-9d16-3d9b7e2d2f0c"). InnerVolumeSpecName "kube-api-access-2dr46". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:27:19.760090 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:19.759998 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dr46\" (UniqueName: \"kubernetes.io/projected/58335c1e-7f13-4d27-9d16-3d9b7e2d2f0c-kube-api-access-2dr46\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:27:20.241019 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:20.240982 2566 generic.go:356] "Generic (PLEG): container finished" podID="58335c1e-7f13-4d27-9d16-3d9b7e2d2f0c" containerID="1f29e76812606afe87a5b64a5446752b5f70058a2a1a0079cc1ec5e0d934cb7f" exitCode=0 Jul 06 18:27:20.241227 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:20.241038 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-clf9h" Jul 06 18:27:20.241227 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:20.241077 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-clf9h" event={"ID":"58335c1e-7f13-4d27-9d16-3d9b7e2d2f0c","Type":"ContainerDied","Data":"1f29e76812606afe87a5b64a5446752b5f70058a2a1a0079cc1ec5e0d934cb7f"} Jul 06 18:27:20.241227 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:20.241125 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-clf9h" event={"ID":"58335c1e-7f13-4d27-9d16-3d9b7e2d2f0c","Type":"ContainerDied","Data":"f84317a398adefda02417af23ed3f2611293dbf59f9bad8909a3cde1ae2a8bf9"} Jul 06 18:27:20.241227 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:20.241147 2566 scope.go:117] "RemoveContainer" containerID="1f29e76812606afe87a5b64a5446752b5f70058a2a1a0079cc1ec5e0d934cb7f" Jul 06 18:27:20.249918 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:20.249903 2566 scope.go:117] "RemoveContainer" containerID="1f29e76812606afe87a5b64a5446752b5f70058a2a1a0079cc1ec5e0d934cb7f" Jul 06 18:27:20.250199 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:27:20.250179 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f29e76812606afe87a5b64a5446752b5f70058a2a1a0079cc1ec5e0d934cb7f\": container with ID starting with 1f29e76812606afe87a5b64a5446752b5f70058a2a1a0079cc1ec5e0d934cb7f not found: ID does not exist" containerID="1f29e76812606afe87a5b64a5446752b5f70058a2a1a0079cc1ec5e0d934cb7f" Jul 06 18:27:20.250270 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:20.250212 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f29e76812606afe87a5b64a5446752b5f70058a2a1a0079cc1ec5e0d934cb7f"} err="failed to get container status \"1f29e76812606afe87a5b64a5446752b5f70058a2a1a0079cc1ec5e0d934cb7f\": rpc error: code = NotFound desc = could not find container \"1f29e76812606afe87a5b64a5446752b5f70058a2a1a0079cc1ec5e0d934cb7f\": container with ID starting with 1f29e76812606afe87a5b64a5446752b5f70058a2a1a0079cc1ec5e0d934cb7f not found: ID does not exist" Jul 06 18:27:20.259519 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:20.259495 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-clf9h"] Jul 06 18:27:20.262776 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:20.262755 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-clf9h"] Jul 06 18:27:21.127219 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:21.127186 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:27:21.228236 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:21.228202 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:27:21.849255 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:21.849222 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58335c1e-7f13-4d27-9d16-3d9b7e2d2f0c" path="/var/lib/kubelet/pods/58335c1e-7f13-4d27-9d16-3d9b7e2d2f0c/volumes" Jul 06 18:27:23.888602 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:23.888569 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:27:25.877161 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:25.877120 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:27:25.976299 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:25.976265 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:27:28.413535 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:28.413504 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-dpmpr"] Jul 06 18:27:28.413949 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:28.413711 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-dpmpr" podUID="bc1d60cd-b84e-49a6-a181-e1ba602eac63" containerName="limitador" containerID="cri-o://c559af0ce9074364cfc52a55c0c3d00af2888cc91656a8868e64f2bb03f136c6" gracePeriod=30 Jul 06 18:27:28.414529 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:28.414510 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-dpmpr" Jul 06 18:27:28.963978 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:28.963954 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-dpmpr" Jul 06 18:27:29.045092 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.045004 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cmz2c\" (UniqueName: \"kubernetes.io/projected/bc1d60cd-b84e-49a6-a181-e1ba602eac63-kube-api-access-cmz2c\") pod \"bc1d60cd-b84e-49a6-a181-e1ba602eac63\" (UID: \"bc1d60cd-b84e-49a6-a181-e1ba602eac63\") " Jul 06 18:27:29.045092 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.045076 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bc1d60cd-b84e-49a6-a181-e1ba602eac63-config-file\") pod \"bc1d60cd-b84e-49a6-a181-e1ba602eac63\" (UID: \"bc1d60cd-b84e-49a6-a181-e1ba602eac63\") " Jul 06 18:27:29.045429 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.045408 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc1d60cd-b84e-49a6-a181-e1ba602eac63-config-file" (OuterVolumeSpecName: "config-file") pod "bc1d60cd-b84e-49a6-a181-e1ba602eac63" (UID: "bc1d60cd-b84e-49a6-a181-e1ba602eac63"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:27:29.047305 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.047275 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc1d60cd-b84e-49a6-a181-e1ba602eac63-kube-api-access-cmz2c" (OuterVolumeSpecName: "kube-api-access-cmz2c") pod "bc1d60cd-b84e-49a6-a181-e1ba602eac63" (UID: "bc1d60cd-b84e-49a6-a181-e1ba602eac63"). InnerVolumeSpecName "kube-api-access-cmz2c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:27:29.145793 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.145757 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmz2c\" (UniqueName: \"kubernetes.io/projected/bc1d60cd-b84e-49a6-a181-e1ba602eac63-kube-api-access-cmz2c\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:27:29.145793 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.145788 2566 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bc1d60cd-b84e-49a6-a181-e1ba602eac63-config-file\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:27:29.289528 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.289493 2566 generic.go:356] "Generic (PLEG): container finished" podID="bc1d60cd-b84e-49a6-a181-e1ba602eac63" containerID="c559af0ce9074364cfc52a55c0c3d00af2888cc91656a8868e64f2bb03f136c6" exitCode=0 Jul 06 18:27:29.289700 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.289541 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-dpmpr" Jul 06 18:27:29.289700 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.289584 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-dpmpr" event={"ID":"bc1d60cd-b84e-49a6-a181-e1ba602eac63","Type":"ContainerDied","Data":"c559af0ce9074364cfc52a55c0c3d00af2888cc91656a8868e64f2bb03f136c6"} Jul 06 18:27:29.289700 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.289641 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-dpmpr" event={"ID":"bc1d60cd-b84e-49a6-a181-e1ba602eac63","Type":"ContainerDied","Data":"9d81cb241b8cf02a7ef224b35246a6d4dabaec71cbe785bf0eca27e700a021a8"} Jul 06 18:27:29.289700 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.289668 2566 scope.go:117] "RemoveContainer" containerID="c559af0ce9074364cfc52a55c0c3d00af2888cc91656a8868e64f2bb03f136c6" Jul 06 18:27:29.298916 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.298894 2566 scope.go:117] "RemoveContainer" containerID="c559af0ce9074364cfc52a55c0c3d00af2888cc91656a8868e64f2bb03f136c6" Jul 06 18:27:29.299185 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:27:29.299166 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c559af0ce9074364cfc52a55c0c3d00af2888cc91656a8868e64f2bb03f136c6\": container with ID starting with c559af0ce9074364cfc52a55c0c3d00af2888cc91656a8868e64f2bb03f136c6 not found: ID does not exist" containerID="c559af0ce9074364cfc52a55c0c3d00af2888cc91656a8868e64f2bb03f136c6" Jul 06 18:27:29.299236 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.299196 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c559af0ce9074364cfc52a55c0c3d00af2888cc91656a8868e64f2bb03f136c6"} err="failed to get container status \"c559af0ce9074364cfc52a55c0c3d00af2888cc91656a8868e64f2bb03f136c6\": rpc error: code = NotFound desc = could not find container \"c559af0ce9074364cfc52a55c0c3d00af2888cc91656a8868e64f2bb03f136c6\": container with ID starting with c559af0ce9074364cfc52a55c0c3d00af2888cc91656a8868e64f2bb03f136c6 not found: ID does not exist" Jul 06 18:27:29.311099 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.311073 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-dpmpr"] Jul 06 18:27:29.316653 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.316630 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-dpmpr"] Jul 06 18:27:29.852906 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.852021 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc1d60cd-b84e-49a6-a181-e1ba602eac63" path="/var/lib/kubelet/pods/bc1d60cd-b84e-49a6-a181-e1ba602eac63/volumes" Jul 06 18:27:29.927817 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.927782 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-cw9k8"] Jul 06 18:27:29.928174 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.928161 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58335c1e-7f13-4d27-9d16-3d9b7e2d2f0c" containerName="authorino" Jul 06 18:27:29.928243 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.928176 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="58335c1e-7f13-4d27-9d16-3d9b7e2d2f0c" containerName="authorino" Jul 06 18:27:29.928243 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.928191 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc1d60cd-b84e-49a6-a181-e1ba602eac63" containerName="limitador" Jul 06 18:27:29.928243 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.928196 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc1d60cd-b84e-49a6-a181-e1ba602eac63" containerName="limitador" Jul 06 18:27:29.928353 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.928258 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc1d60cd-b84e-49a6-a181-e1ba602eac63" containerName="limitador" Jul 06 18:27:29.928353 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.928268 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="58335c1e-7f13-4d27-9d16-3d9b7e2d2f0c" containerName="authorino" Jul 06 18:27:29.932808 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.932789 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-cw9k8" Jul 06 18:27:29.935992 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.935960 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 06 18:27:29.936891 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.936851 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qv6d8\"" Jul 06 18:27:29.938890 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.938844 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-cw9k8"] Jul 06 18:27:29.953918 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.953855 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5bd217af-1704-4e03-8ebe-8c300c47dbff-data\") pod \"postgres-69fb4594cf-cw9k8\" (UID: \"5bd217af-1704-4e03-8ebe-8c300c47dbff\") " pod="opendatahub/postgres-69fb4594cf-cw9k8" Jul 06 18:27:29.954084 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.953940 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68ll4\" (UniqueName: \"kubernetes.io/projected/5bd217af-1704-4e03-8ebe-8c300c47dbff-kube-api-access-68ll4\") pod \"postgres-69fb4594cf-cw9k8\" (UID: \"5bd217af-1704-4e03-8ebe-8c300c47dbff\") " pod="opendatahub/postgres-69fb4594cf-cw9k8" Jul 06 18:27:29.959043 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:29.959016 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:27:30.054992 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:30.054955 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5bd217af-1704-4e03-8ebe-8c300c47dbff-data\") pod \"postgres-69fb4594cf-cw9k8\" (UID: \"5bd217af-1704-4e03-8ebe-8c300c47dbff\") " pod="opendatahub/postgres-69fb4594cf-cw9k8" Jul 06 18:27:30.055173 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:30.055040 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68ll4\" (UniqueName: \"kubernetes.io/projected/5bd217af-1704-4e03-8ebe-8c300c47dbff-kube-api-access-68ll4\") pod \"postgres-69fb4594cf-cw9k8\" (UID: \"5bd217af-1704-4e03-8ebe-8c300c47dbff\") " pod="opendatahub/postgres-69fb4594cf-cw9k8" Jul 06 18:27:30.055360 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:30.055339 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5bd217af-1704-4e03-8ebe-8c300c47dbff-data\") pod \"postgres-69fb4594cf-cw9k8\" (UID: \"5bd217af-1704-4e03-8ebe-8c300c47dbff\") " pod="opendatahub/postgres-69fb4594cf-cw9k8" Jul 06 18:27:30.063732 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:30.063706 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68ll4\" (UniqueName: \"kubernetes.io/projected/5bd217af-1704-4e03-8ebe-8c300c47dbff-kube-api-access-68ll4\") pod \"postgres-69fb4594cf-cw9k8\" (UID: \"5bd217af-1704-4e03-8ebe-8c300c47dbff\") " pod="opendatahub/postgres-69fb4594cf-cw9k8" Jul 06 18:27:30.245250 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:30.245219 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-cw9k8" Jul 06 18:27:30.386174 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:30.386147 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-cw9k8"] Jul 06 18:27:30.387607 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:27:30.387573 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bd217af_1704_4e03_8ebe_8c300c47dbff.slice/crio-8535a2cb9c856e20fe80f0023731f5551311e509fa4a2069ec745941b4e560a7 WatchSource:0}: Error finding container 8535a2cb9c856e20fe80f0023731f5551311e509fa4a2069ec745941b4e560a7: Status 404 returned error can't find the container with id 8535a2cb9c856e20fe80f0023731f5551311e509fa4a2069ec745941b4e560a7 Jul 06 18:27:31.307156 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:31.307116 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-cw9k8" event={"ID":"5bd217af-1704-4e03-8ebe-8c300c47dbff","Type":"ContainerStarted","Data":"8535a2cb9c856e20fe80f0023731f5551311e509fa4a2069ec745941b4e560a7"} Jul 06 18:27:40.369399 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:40.369360 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-cw9k8" event={"ID":"5bd217af-1704-4e03-8ebe-8c300c47dbff","Type":"ContainerStarted","Data":"0618eba2146d1ca41912225556f347d254478a879502db0181a01fa6dfcf5957"} Jul 06 18:27:40.369815 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:40.369493 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-cw9k8" Jul 06 18:27:46.405979 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:46.405943 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-cw9k8" Jul 06 18:27:46.425974 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:46.425923 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-cw9k8" podStartSLOduration=8.086537213 podStartE2EDuration="17.425907731s" totalImagesPullingTime="9.339370518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:27:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:27:40.386241283 +0000 UTC m=+477.112670240" watchObservedRunningTime="2026-07-06 18:27:46.425907731 +0000 UTC m=+483.152336685" Jul 06 18:27:46.766645 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:46.766549 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:27:46.859902 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:46.859797 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:27:47.141371 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:47.141312 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:27:48.451617 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:48.451579 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks"] Jul 06 18:27:48.457825 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:48.457804 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks" Jul 06 18:27:48.460884 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:48.460843 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rqp65\"" Jul 06 18:27:48.461009 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:48.460888 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 18:27:48.462105 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:48.462084 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 18:27:48.466436 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:48.466411 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks"] Jul 06 18:27:48.520052 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:48.520009 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1038c2b-06ca-4c5d-b373-372fa2bbcbb1-bundle\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks\" (UID: \"e1038c2b-06ca-4c5d-b373-372fa2bbcbb1\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks" Jul 06 18:27:48.520052 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:48.520055 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1038c2b-06ca-4c5d-b373-372fa2bbcbb1-util\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks\" (UID: \"e1038c2b-06ca-4c5d-b373-372fa2bbcbb1\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks" Jul 06 18:27:48.520288 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:48.520074 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q9ch\" (UniqueName: \"kubernetes.io/projected/e1038c2b-06ca-4c5d-b373-372fa2bbcbb1-kube-api-access-7q9ch\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks\" (UID: \"e1038c2b-06ca-4c5d-b373-372fa2bbcbb1\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks" Jul 06 18:27:48.621165 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:48.621126 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1038c2b-06ca-4c5d-b373-372fa2bbcbb1-bundle\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks\" (UID: \"e1038c2b-06ca-4c5d-b373-372fa2bbcbb1\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks" Jul 06 18:27:48.621165 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:48.621171 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1038c2b-06ca-4c5d-b373-372fa2bbcbb1-util\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks\" (UID: \"e1038c2b-06ca-4c5d-b373-372fa2bbcbb1\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks" Jul 06 18:27:48.621409 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:48.621189 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q9ch\" (UniqueName: \"kubernetes.io/projected/e1038c2b-06ca-4c5d-b373-372fa2bbcbb1-kube-api-access-7q9ch\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks\" (UID: \"e1038c2b-06ca-4c5d-b373-372fa2bbcbb1\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks" Jul 06 18:27:48.621484 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:48.621461 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1038c2b-06ca-4c5d-b373-372fa2bbcbb1-bundle\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks\" (UID: \"e1038c2b-06ca-4c5d-b373-372fa2bbcbb1\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks" Jul 06 18:27:48.621552 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:48.621486 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1038c2b-06ca-4c5d-b373-372fa2bbcbb1-util\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks\" (UID: \"e1038c2b-06ca-4c5d-b373-372fa2bbcbb1\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks" Jul 06 18:27:48.630309 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:48.630282 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q9ch\" (UniqueName: \"kubernetes.io/projected/e1038c2b-06ca-4c5d-b373-372fa2bbcbb1-kube-api-access-7q9ch\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks\" (UID: \"e1038c2b-06ca-4c5d-b373-372fa2bbcbb1\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks" Jul 06 18:27:48.768409 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:48.768322 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks" Jul 06 18:27:48.908079 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:48.908052 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks"] Jul 06 18:27:48.909700 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:27:48.909676 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1038c2b_06ca_4c5d_b373_372fa2bbcbb1.slice/crio-cbab3897eeb329d19d3c3feb320b680cadba4cc33239aaa613cf230073d229dd WatchSource:0}: Error finding container cbab3897eeb329d19d3c3feb320b680cadba4cc33239aaa613cf230073d229dd: Status 404 returned error can't find the container with id cbab3897eeb329d19d3c3feb320b680cadba4cc33239aaa613cf230073d229dd Jul 06 18:27:49.417456 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:49.417419 2566 generic.go:356] "Generic (PLEG): container finished" podID="e1038c2b-06ca-4c5d-b373-372fa2bbcbb1" containerID="c7fba3d613b9a852e934a4f8114b8a210ffde4f8511e76315045239a3f8ead9d" exitCode=0 Jul 06 18:27:49.417657 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:49.417480 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks" event={"ID":"e1038c2b-06ca-4c5d-b373-372fa2bbcbb1","Type":"ContainerDied","Data":"c7fba3d613b9a852e934a4f8114b8a210ffde4f8511e76315045239a3f8ead9d"} Jul 06 18:27:49.417657 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:49.417525 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks" event={"ID":"e1038c2b-06ca-4c5d-b373-372fa2bbcbb1","Type":"ContainerStarted","Data":"cbab3897eeb329d19d3c3feb320b680cadba4cc33239aaa613cf230073d229dd"} Jul 06 18:27:51.431670 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:51.431636 2566 generic.go:356] "Generic (PLEG): container finished" podID="e1038c2b-06ca-4c5d-b373-372fa2bbcbb1" containerID="0fd37f9280fdc6341a8335a9e998d3825ccf24fd8f77d1ba736b11c80f7dd90d" exitCode=0 Jul 06 18:27:51.432153 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:51.431694 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks" event={"ID":"e1038c2b-06ca-4c5d-b373-372fa2bbcbb1","Type":"ContainerDied","Data":"0fd37f9280fdc6341a8335a9e998d3825ccf24fd8f77d1ba736b11c80f7dd90d"} Jul 06 18:27:52.439384 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:52.439349 2566 generic.go:356] "Generic (PLEG): container finished" podID="e1038c2b-06ca-4c5d-b373-372fa2bbcbb1" containerID="8843e9776d46e6ff55e78c7f54c3ef7ccce57a81fa3b2d1f424fa12e65bce283" exitCode=0 Jul 06 18:27:52.439793 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:52.439400 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks" event={"ID":"e1038c2b-06ca-4c5d-b373-372fa2bbcbb1","Type":"ContainerDied","Data":"8843e9776d46e6ff55e78c7f54c3ef7ccce57a81fa3b2d1f424fa12e65bce283"} Jul 06 18:27:53.698841 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:53.698766 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks" Jul 06 18:27:53.769252 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:53.769215 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1038c2b-06ca-4c5d-b373-372fa2bbcbb1-util\") pod \"e1038c2b-06ca-4c5d-b373-372fa2bbcbb1\" (UID: \"e1038c2b-06ca-4c5d-b373-372fa2bbcbb1\") " Jul 06 18:27:53.769252 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:53.769259 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7q9ch\" (UniqueName: \"kubernetes.io/projected/e1038c2b-06ca-4c5d-b373-372fa2bbcbb1-kube-api-access-7q9ch\") pod \"e1038c2b-06ca-4c5d-b373-372fa2bbcbb1\" (UID: \"e1038c2b-06ca-4c5d-b373-372fa2bbcbb1\") " Jul 06 18:27:53.769480 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:53.769343 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1038c2b-06ca-4c5d-b373-372fa2bbcbb1-bundle\") pod \"e1038c2b-06ca-4c5d-b373-372fa2bbcbb1\" (UID: \"e1038c2b-06ca-4c5d-b373-372fa2bbcbb1\") " Jul 06 18:27:53.769902 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:53.769852 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1038c2b-06ca-4c5d-b373-372fa2bbcbb1-bundle" (OuterVolumeSpecName: "bundle") pod "e1038c2b-06ca-4c5d-b373-372fa2bbcbb1" (UID: "e1038c2b-06ca-4c5d-b373-372fa2bbcbb1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:27:53.771603 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:53.771582 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1038c2b-06ca-4c5d-b373-372fa2bbcbb1-kube-api-access-7q9ch" (OuterVolumeSpecName: "kube-api-access-7q9ch") pod "e1038c2b-06ca-4c5d-b373-372fa2bbcbb1" (UID: "e1038c2b-06ca-4c5d-b373-372fa2bbcbb1"). InnerVolumeSpecName "kube-api-access-7q9ch". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:27:53.774257 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:53.774226 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1038c2b-06ca-4c5d-b373-372fa2bbcbb1-util" (OuterVolumeSpecName: "util") pod "e1038c2b-06ca-4c5d-b373-372fa2bbcbb1" (UID: "e1038c2b-06ca-4c5d-b373-372fa2bbcbb1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:27:53.870044 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:53.870014 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1038c2b-06ca-4c5d-b373-372fa2bbcbb1-bundle\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:27:53.870044 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:53.870046 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1038c2b-06ca-4c5d-b373-372fa2bbcbb1-util\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:27:53.870251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:53.870060 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7q9ch\" (UniqueName: \"kubernetes.io/projected/e1038c2b-06ca-4c5d-b373-372fa2bbcbb1-kube-api-access-7q9ch\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:27:54.451671 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:54.451634 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks" event={"ID":"e1038c2b-06ca-4c5d-b373-372fa2bbcbb1","Type":"ContainerDied","Data":"cbab3897eeb329d19d3c3feb320b680cadba4cc33239aaa613cf230073d229dd"} Jul 06 18:27:54.451671 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:54.451657 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c88ssks" Jul 06 18:27:54.451671 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:27:54.451680 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cbab3897eeb329d19d3c3feb320b680cadba4cc33239aaa613cf230073d229dd" Jul 06 18:28:01.880657 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:01.880617 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/keycloak-operator-d8b9d7b4d-wv9bd"] Jul 06 18:28:01.881106 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:01.881011 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1038c2b-06ca-4c5d-b373-372fa2bbcbb1" containerName="util" Jul 06 18:28:01.881106 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:01.881024 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1038c2b-06ca-4c5d-b373-372fa2bbcbb1" containerName="util" Jul 06 18:28:01.881106 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:01.881032 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1038c2b-06ca-4c5d-b373-372fa2bbcbb1" containerName="pull" Jul 06 18:28:01.881106 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:01.881037 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1038c2b-06ca-4c5d-b373-372fa2bbcbb1" containerName="pull" Jul 06 18:28:01.881106 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:01.881059 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1038c2b-06ca-4c5d-b373-372fa2bbcbb1" containerName="extract" Jul 06 18:28:01.881106 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:01.881065 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1038c2b-06ca-4c5d-b373-372fa2bbcbb1" containerName="extract" Jul 06 18:28:01.881294 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:01.881118 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1038c2b-06ca-4c5d-b373-372fa2bbcbb1" containerName="extract" Jul 06 18:28:01.883361 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:01.883344 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-d8b9d7b4d-wv9bd" Jul 06 18:28:01.886426 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:01.886394 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jul 06 18:28:01.887007 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:01.886655 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jul 06 18:28:01.887141 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:01.887102 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"keycloak-operator-dockercfg-ljjnh\"" Jul 06 18:28:01.896249 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:01.896214 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-d8b9d7b4d-wv9bd"] Jul 06 18:28:02.041898 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:02.041829 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7v6w\" (UniqueName: \"kubernetes.io/projected/3e0a436c-302b-410c-a7a8-b82fa1012d4a-kube-api-access-q7v6w\") pod \"keycloak-operator-d8b9d7b4d-wv9bd\" (UID: \"3e0a436c-302b-410c-a7a8-b82fa1012d4a\") " pod="keycloak-system/keycloak-operator-d8b9d7b4d-wv9bd" Jul 06 18:28:02.142930 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:02.142891 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7v6w\" (UniqueName: \"kubernetes.io/projected/3e0a436c-302b-410c-a7a8-b82fa1012d4a-kube-api-access-q7v6w\") pod \"keycloak-operator-d8b9d7b4d-wv9bd\" (UID: \"3e0a436c-302b-410c-a7a8-b82fa1012d4a\") " pod="keycloak-system/keycloak-operator-d8b9d7b4d-wv9bd" Jul 06 18:28:02.154878 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:02.154843 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7v6w\" (UniqueName: \"kubernetes.io/projected/3e0a436c-302b-410c-a7a8-b82fa1012d4a-kube-api-access-q7v6w\") pod \"keycloak-operator-d8b9d7b4d-wv9bd\" (UID: \"3e0a436c-302b-410c-a7a8-b82fa1012d4a\") " pod="keycloak-system/keycloak-operator-d8b9d7b4d-wv9bd" Jul 06 18:28:02.201991 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:02.201954 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-d8b9d7b4d-wv9bd" Jul 06 18:28:02.339379 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:02.339354 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-d8b9d7b4d-wv9bd"] Jul 06 18:28:02.340616 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:28:02.340586 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e0a436c_302b_410c_a7a8_b82fa1012d4a.slice/crio-e22c814cc3c34eae581ff795d86f758cc4c16dd4d98eb552141721f53ea5635b WatchSource:0}: Error finding container e22c814cc3c34eae581ff795d86f758cc4c16dd4d98eb552141721f53ea5635b: Status 404 returned error can't find the container with id e22c814cc3c34eae581ff795d86f758cc4c16dd4d98eb552141721f53ea5635b Jul 06 18:28:02.501533 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:02.501417 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-d8b9d7b4d-wv9bd" event={"ID":"3e0a436c-302b-410c-a7a8-b82fa1012d4a","Type":"ContainerStarted","Data":"e22c814cc3c34eae581ff795d86f758cc4c16dd4d98eb552141721f53ea5635b"} Jul 06 18:28:08.541747 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:08.541712 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-d8b9d7b4d-wv9bd" event={"ID":"3e0a436c-302b-410c-a7a8-b82fa1012d4a","Type":"ContainerStarted","Data":"6b752aa157ef8dc2c505bce767a8fcc1d4b27aa807e6f87cefeb206d2d52113f"} Jul 06 18:28:08.563768 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:08.563717 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/keycloak-operator-d8b9d7b4d-wv9bd" podStartSLOduration=1.5462298030000001 podStartE2EDuration="7.563703134s" totalImagesPullingTime="6.017473331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:28:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:28:08.562740022 +0000 UTC m=+505.289168981" watchObservedRunningTime="2026-07-06 18:28:08.563703134 +0000 UTC m=+505.290132092" Jul 06 18:28:17.390387 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:17.390347 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:28:17.495470 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:17.495431 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:28:47.903699 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:47.903622 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:28:48.087778 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.087745 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:28:48.149634 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.149604 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7t2m9"] Jul 06 18:28:48.157110 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.157048 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7t2m9" Jul 06 18:28:48.162184 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.162156 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7t2m9"] Jul 06 18:28:48.165426 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.165399 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-5bwhk\"" Jul 06 18:28:48.284238 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.284189 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x24n\" (UniqueName: \"kubernetes.io/projected/e3309493-9db7-44b4-8535-7004d0623a80-kube-api-access-9x24n\") pod \"authorino-5d4b67c446-7t2m9\" (UID: \"e3309493-9db7-44b4-8535-7004d0623a80\") " pod="kuadrant-system/authorino-5d4b67c446-7t2m9" Jul 06 18:28:48.381182 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.381136 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7t2m9"] Jul 06 18:28:48.381486 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:28:48.381461 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-9x24n], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-7t2m9" podUID="e3309493-9db7-44b4-8535-7004d0623a80" Jul 06 18:28:48.385152 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.385130 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x24n\" (UniqueName: \"kubernetes.io/projected/e3309493-9db7-44b4-8535-7004d0623a80-kube-api-access-9x24n\") pod \"authorino-5d4b67c446-7t2m9\" (UID: \"e3309493-9db7-44b4-8535-7004d0623a80\") " pod="kuadrant-system/authorino-5d4b67c446-7t2m9" Jul 06 18:28:48.407650 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.407586 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x24n\" (UniqueName: \"kubernetes.io/projected/e3309493-9db7-44b4-8535-7004d0623a80-kube-api-access-9x24n\") pod \"authorino-5d4b67c446-7t2m9\" (UID: \"e3309493-9db7-44b4-8535-7004d0623a80\") " pod="kuadrant-system/authorino-5d4b67c446-7t2m9" Jul 06 18:28:48.418739 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.418707 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7db7d464d9-2w7xf"] Jul 06 18:28:48.421203 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.421188 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7db7d464d9-2w7xf" Jul 06 18:28:48.435684 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.435660 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7db7d464d9-2w7xf"] Jul 06 18:28:48.586318 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.586285 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwmq2\" (UniqueName: \"kubernetes.io/projected/64820f74-115d-4fec-81a8-e45d261d82b6-kube-api-access-qwmq2\") pod \"authorino-7db7d464d9-2w7xf\" (UID: \"64820f74-115d-4fec-81a8-e45d261d82b6\") " pod="kuadrant-system/authorino-7db7d464d9-2w7xf" Jul 06 18:28:48.642554 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.642526 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7db7d464d9-2w7xf"] Jul 06 18:28:48.642832 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:28:48.642812 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-qwmq2], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-7db7d464d9-2w7xf" podUID="64820f74-115d-4fec-81a8-e45d261d82b6" Jul 06 18:28:48.686952 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.686851 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwmq2\" (UniqueName: \"kubernetes.io/projected/64820f74-115d-4fec-81a8-e45d261d82b6-kube-api-access-qwmq2\") pod \"authorino-7db7d464d9-2w7xf\" (UID: \"64820f74-115d-4fec-81a8-e45d261d82b6\") " pod="kuadrant-system/authorino-7db7d464d9-2w7xf" Jul 06 18:28:48.692181 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.692161 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8c9f6f57c-52jbr"] Jul 06 18:28:48.694789 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.694771 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8c9f6f57c-52jbr" Jul 06 18:28:48.709776 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.709757 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 06 18:28:48.711688 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.711665 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwmq2\" (UniqueName: \"kubernetes.io/projected/64820f74-115d-4fec-81a8-e45d261d82b6-kube-api-access-qwmq2\") pod \"authorino-7db7d464d9-2w7xf\" (UID: \"64820f74-115d-4fec-81a8-e45d261d82b6\") " pod="kuadrant-system/authorino-7db7d464d9-2w7xf" Jul 06 18:28:48.720892 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.720554 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8c9f6f57c-52jbr"] Jul 06 18:28:48.754172 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.754141 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7t2m9" Jul 06 18:28:48.754324 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.754141 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7db7d464d9-2w7xf" Jul 06 18:28:48.761285 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.761265 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7t2m9" Jul 06 18:28:48.763642 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.763625 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7db7d464d9-2w7xf" Jul 06 18:28:48.788119 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.788092 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7bc983f8-7cac-4d08-baa2-5ff0816784fa-tls-cert\") pod \"authorino-8c9f6f57c-52jbr\" (UID: \"7bc983f8-7cac-4d08-baa2-5ff0816784fa\") " pod="kuadrant-system/authorino-8c9f6f57c-52jbr" Jul 06 18:28:48.788231 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.788201 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rp242\" (UniqueName: \"kubernetes.io/projected/7bc983f8-7cac-4d08-baa2-5ff0816784fa-kube-api-access-rp242\") pod \"authorino-8c9f6f57c-52jbr\" (UID: \"7bc983f8-7cac-4d08-baa2-5ff0816784fa\") " pod="kuadrant-system/authorino-8c9f6f57c-52jbr" Jul 06 18:28:48.888678 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.888647 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwmq2\" (UniqueName: \"kubernetes.io/projected/64820f74-115d-4fec-81a8-e45d261d82b6-kube-api-access-qwmq2\") pod \"64820f74-115d-4fec-81a8-e45d261d82b6\" (UID: \"64820f74-115d-4fec-81a8-e45d261d82b6\") " Jul 06 18:28:48.888894 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.888721 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9x24n\" (UniqueName: \"kubernetes.io/projected/e3309493-9db7-44b4-8535-7004d0623a80-kube-api-access-9x24n\") pod \"e3309493-9db7-44b4-8535-7004d0623a80\" (UID: \"e3309493-9db7-44b4-8535-7004d0623a80\") " Jul 06 18:28:48.888954 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.888906 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rp242\" (UniqueName: \"kubernetes.io/projected/7bc983f8-7cac-4d08-baa2-5ff0816784fa-kube-api-access-rp242\") pod \"authorino-8c9f6f57c-52jbr\" (UID: \"7bc983f8-7cac-4d08-baa2-5ff0816784fa\") " pod="kuadrant-system/authorino-8c9f6f57c-52jbr" Jul 06 18:28:48.888954 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.888951 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7bc983f8-7cac-4d08-baa2-5ff0816784fa-tls-cert\") pod \"authorino-8c9f6f57c-52jbr\" (UID: \"7bc983f8-7cac-4d08-baa2-5ff0816784fa\") " pod="kuadrant-system/authorino-8c9f6f57c-52jbr" Jul 06 18:28:48.891050 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.891020 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64820f74-115d-4fec-81a8-e45d261d82b6-kube-api-access-qwmq2" (OuterVolumeSpecName: "kube-api-access-qwmq2") pod "64820f74-115d-4fec-81a8-e45d261d82b6" (UID: "64820f74-115d-4fec-81a8-e45d261d82b6"). InnerVolumeSpecName "kube-api-access-qwmq2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:28:48.891170 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.891055 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3309493-9db7-44b4-8535-7004d0623a80-kube-api-access-9x24n" (OuterVolumeSpecName: "kube-api-access-9x24n") pod "e3309493-9db7-44b4-8535-7004d0623a80" (UID: "e3309493-9db7-44b4-8535-7004d0623a80"). InnerVolumeSpecName "kube-api-access-9x24n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:28:48.891414 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.891394 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7bc983f8-7cac-4d08-baa2-5ff0816784fa-tls-cert\") pod \"authorino-8c9f6f57c-52jbr\" (UID: \"7bc983f8-7cac-4d08-baa2-5ff0816784fa\") " pod="kuadrant-system/authorino-8c9f6f57c-52jbr" Jul 06 18:28:48.906678 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.906649 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rp242\" (UniqueName: \"kubernetes.io/projected/7bc983f8-7cac-4d08-baa2-5ff0816784fa-kube-api-access-rp242\") pod \"authorino-8c9f6f57c-52jbr\" (UID: \"7bc983f8-7cac-4d08-baa2-5ff0816784fa\") " pod="kuadrant-system/authorino-8c9f6f57c-52jbr" Jul 06 18:28:48.990135 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.990046 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwmq2\" (UniqueName: \"kubernetes.io/projected/64820f74-115d-4fec-81a8-e45d261d82b6-kube-api-access-qwmq2\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:28:48.990135 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:48.990082 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9x24n\" (UniqueName: \"kubernetes.io/projected/e3309493-9db7-44b4-8535-7004d0623a80-kube-api-access-9x24n\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:28:49.003810 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:49.003778 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8c9f6f57c-52jbr" Jul 06 18:28:49.157539 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:28:49.157511 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7bc983f8_7cac_4d08_baa2_5ff0816784fa.slice/crio-00554d7b270cdf4a9effbc15eb32f654a9746ae3a59a95e370863ed3083a2384 WatchSource:0}: Error finding container 00554d7b270cdf4a9effbc15eb32f654a9746ae3a59a95e370863ed3083a2384: Status 404 returned error can't find the container with id 00554d7b270cdf4a9effbc15eb32f654a9746ae3a59a95e370863ed3083a2384 Jul 06 18:28:49.159179 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:49.159156 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8c9f6f57c-52jbr"] Jul 06 18:28:49.761060 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:49.761019 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8c9f6f57c-52jbr" event={"ID":"7bc983f8-7cac-4d08-baa2-5ff0816784fa","Type":"ContainerStarted","Data":"36745e8c0353dceb8ca4132efa9b05ee6f2f2b87e64b5502ca7008344eda48ca"} Jul 06 18:28:49.761251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:49.761071 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8c9f6f57c-52jbr" event={"ID":"7bc983f8-7cac-4d08-baa2-5ff0816784fa","Type":"ContainerStarted","Data":"00554d7b270cdf4a9effbc15eb32f654a9746ae3a59a95e370863ed3083a2384"} Jul 06 18:28:49.761251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:49.761090 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7t2m9" Jul 06 18:28:49.761251 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:49.761113 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7db7d464d9-2w7xf" Jul 06 18:28:49.802310 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:49.802255 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-8c9f6f57c-52jbr" podStartSLOduration=1.446572254 podStartE2EDuration="1.802239828s" totalImagesPullingTime="355.667574ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:28:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:28:49.801494631 +0000 UTC m=+546.527923590" watchObservedRunningTime="2026-07-06 18:28:49.802239828 +0000 UTC m=+546.528668786" Jul 06 18:28:49.855493 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:49.855458 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7t2m9"] Jul 06 18:28:49.857762 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:49.857732 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7t2m9"] Jul 06 18:28:49.902136 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:49.902098 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7db7d464d9-2w7xf"] Jul 06 18:28:49.906522 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:49.906493 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7db7d464d9-2w7xf"] Jul 06 18:28:51.849538 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:51.849503 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64820f74-115d-4fec-81a8-e45d261d82b6" path="/var/lib/kubelet/pods/64820f74-115d-4fec-81a8-e45d261d82b6/volumes" Jul 06 18:28:51.849955 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:51.849735 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3309493-9db7-44b4-8535-7004d0623a80" path="/var/lib/kubelet/pods/e3309493-9db7-44b4-8535-7004d0623a80/volumes" Jul 06 18:28:54.184945 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:54.184909 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:28:54.281332 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:54.281294 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:28:54.373394 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:54.373360 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:28:54.699640 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:54.699614 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:28:58.145670 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:58.145634 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:28:58.242247 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:28:58.242214 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:29:22.519657 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:29:22.519624 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:29:28.695569 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:29:28.695517 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:29:29.155744 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:29:29.155717 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:29:29.354115 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:29:29.354085 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:29:29.665150 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:29:29.665112 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:29:29.765332 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:29:29.765304 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:29:38.021340 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:29:38.021300 2566 generic.go:356] "Generic (PLEG): container finished" podID="82159f17-c966-4747-aec4-1dd75e8726d5" containerID="88afcb9a738646b80a832cecee909adfc047b277d99d642499c98c8d58d81fc1" exitCode=0 Jul 06 18:29:38.021756 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:29:38.021379 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" event={"ID":"82159f17-c966-4747-aec4-1dd75e8726d5","Type":"ContainerDied","Data":"88afcb9a738646b80a832cecee909adfc047b277d99d642499c98c8d58d81fc1"} Jul 06 18:29:38.021825 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:29:38.021810 2566 scope.go:117] "RemoveContainer" containerID="88afcb9a738646b80a832cecee909adfc047b277d99d642499c98c8d58d81fc1" Jul 06 18:29:38.022828 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:29:38.022799 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:29:38.022906 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:29:38.022887 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:29:38.022956 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:29:38.022923 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:29:39.029236 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:29:39.029199 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" event={"ID":"82159f17-c966-4747-aec4-1dd75e8726d5","Type":"ContainerStarted","Data":"f4a1756560251c336f74998d5f191ca7acaace032bf62047c6d0572094bf03a9"} Jul 06 18:29:39.199277 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:29:39.199235 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:29:39.199477 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:29:39.199369 2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-p5nb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 06 18:29:39.199477 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:29:39.199424 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" podUID="82159f17-c966-4747-aec4-1dd75e8726d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 06 18:29:40.198417 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:29:40.198368 2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-p5nb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 06 18:29:40.199006 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:29:40.198439 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" podUID="82159f17-c966-4747-aec4-1dd75e8726d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 06 18:29:41.198733 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:29:41.198693 2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-p5nb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 06 18:29:41.199354 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:29:41.199321 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" podUID="82159f17-c966-4747-aec4-1dd75e8726d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 06 18:29:42.198666 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:29:42.198614 2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-p5nb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 06 18:29:42.198990 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:29:42.198957 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" podUID="82159f17-c966-4747-aec4-1dd75e8726d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 06 18:29:43.183019 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:29:43.182986 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:29:43.202651 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:29:43.202624 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:29:43.736270 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:29:43.736228 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffzjr_1ede441d-79f2-4365-8a08-403bf043ef73/ovn-acl-logging/0.log" Jul 06 18:29:43.737455 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:29:43.737436 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffzjr_1ede441d-79f2-4365-8a08-403bf043ef73/ovn-acl-logging/0.log" Jul 06 18:29:44.057446 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:29:44.057359 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-p5nb4" Jul 06 18:29:54.418512 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:29:54.418477 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:30:17.453817 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:17.453727 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f9f7b785-p7x7q"] Jul 06 18:30:17.457418 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:17.457391 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f9f7b785-p7x7q" Jul 06 18:30:17.460682 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:17.460647 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\"" Jul 06 18:30:17.464137 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:17.464110 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f9f7b785-p7x7q"] Jul 06 18:30:17.598101 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:17.598050 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6ecbccc2-d982-4e9c-b6ee-2c04f32ae723-tls-cert\") pod \"authorino-6f9f7b785-p7x7q\" (UID: \"6ecbccc2-d982-4e9c-b6ee-2c04f32ae723\") " pod="kuadrant-system/authorino-6f9f7b785-p7x7q" Jul 06 18:30:17.598282 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:17.598155 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/6ecbccc2-d982-4e9c-b6ee-2c04f32ae723-oidc-ca\") pod \"authorino-6f9f7b785-p7x7q\" (UID: \"6ecbccc2-d982-4e9c-b6ee-2c04f32ae723\") " pod="kuadrant-system/authorino-6f9f7b785-p7x7q" Jul 06 18:30:17.598282 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:17.598183 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hssw7\" (UniqueName: \"kubernetes.io/projected/6ecbccc2-d982-4e9c-b6ee-2c04f32ae723-kube-api-access-hssw7\") pod \"authorino-6f9f7b785-p7x7q\" (UID: \"6ecbccc2-d982-4e9c-b6ee-2c04f32ae723\") " pod="kuadrant-system/authorino-6f9f7b785-p7x7q" Jul 06 18:30:17.698758 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:17.698722 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6ecbccc2-d982-4e9c-b6ee-2c04f32ae723-tls-cert\") pod \"authorino-6f9f7b785-p7x7q\" (UID: \"6ecbccc2-d982-4e9c-b6ee-2c04f32ae723\") " pod="kuadrant-system/authorino-6f9f7b785-p7x7q" Jul 06 18:30:17.698953 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:17.698790 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/6ecbccc2-d982-4e9c-b6ee-2c04f32ae723-oidc-ca\") pod \"authorino-6f9f7b785-p7x7q\" (UID: \"6ecbccc2-d982-4e9c-b6ee-2c04f32ae723\") " pod="kuadrant-system/authorino-6f9f7b785-p7x7q" Jul 06 18:30:17.698953 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:17.698841 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hssw7\" (UniqueName: \"kubernetes.io/projected/6ecbccc2-d982-4e9c-b6ee-2c04f32ae723-kube-api-access-hssw7\") pod \"authorino-6f9f7b785-p7x7q\" (UID: \"6ecbccc2-d982-4e9c-b6ee-2c04f32ae723\") " pod="kuadrant-system/authorino-6f9f7b785-p7x7q" Jul 06 18:30:17.699611 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:17.699592 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/6ecbccc2-d982-4e9c-b6ee-2c04f32ae723-oidc-ca\") pod \"authorino-6f9f7b785-p7x7q\" (UID: \"6ecbccc2-d982-4e9c-b6ee-2c04f32ae723\") " pod="kuadrant-system/authorino-6f9f7b785-p7x7q" Jul 06 18:30:17.701320 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:17.701301 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6ecbccc2-d982-4e9c-b6ee-2c04f32ae723-tls-cert\") pod \"authorino-6f9f7b785-p7x7q\" (UID: \"6ecbccc2-d982-4e9c-b6ee-2c04f32ae723\") " pod="kuadrant-system/authorino-6f9f7b785-p7x7q" Jul 06 18:30:17.707145 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:17.707081 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hssw7\" (UniqueName: \"kubernetes.io/projected/6ecbccc2-d982-4e9c-b6ee-2c04f32ae723-kube-api-access-hssw7\") pod \"authorino-6f9f7b785-p7x7q\" (UID: \"6ecbccc2-d982-4e9c-b6ee-2c04f32ae723\") " pod="kuadrant-system/authorino-6f9f7b785-p7x7q" Jul 06 18:30:17.768298 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:17.768264 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f9f7b785-p7x7q" Jul 06 18:30:17.909923 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:17.909897 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f9f7b785-p7x7q"] Jul 06 18:30:17.911188 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:30:17.911158 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6ecbccc2_d982_4e9c_b6ee_2c04f32ae723.slice/crio-8a29abd83187e43201208966ac89ab189008a25f46c9cac2122ca8445d58abdd WatchSource:0}: Error finding container 8a29abd83187e43201208966ac89ab189008a25f46c9cac2122ca8445d58abdd: Status 404 returned error can't find the container with id 8a29abd83187e43201208966ac89ab189008a25f46c9cac2122ca8445d58abdd Jul 06 18:30:17.912502 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:17.912484 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 18:30:18.248553 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:18.248514 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f9f7b785-p7x7q" event={"ID":"6ecbccc2-d982-4e9c-b6ee-2c04f32ae723","Type":"ContainerStarted","Data":"8a29abd83187e43201208966ac89ab189008a25f46c9cac2122ca8445d58abdd"} Jul 06 18:30:19.255726 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:19.255688 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f9f7b785-p7x7q" event={"ID":"6ecbccc2-d982-4e9c-b6ee-2c04f32ae723","Type":"ContainerStarted","Data":"df789e12966eee0fe78c4364081a5b924ef0a8b0446644f9a7e9253ebc9874f1"} Jul 06 18:30:19.283029 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:19.282979 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6f9f7b785-p7x7q" podStartSLOduration=1.894578637 podStartE2EDuration="2.282964058s" totalImagesPullingTime="388.385421ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:30:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:30:19.280494668 +0000 UTC m=+636.006923626" watchObservedRunningTime="2026-07-06 18:30:19.282964058 +0000 UTC m=+636.009393018" Jul 06 18:30:19.308328 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:19.308295 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8c9f6f57c-52jbr"] Jul 06 18:30:19.308527 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:19.308481 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-8c9f6f57c-52jbr" podUID="7bc983f8-7cac-4d08-baa2-5ff0816784fa" containerName="authorino" containerID="cri-o://36745e8c0353dceb8ca4132efa9b05ee6f2f2b87e64b5502ca7008344eda48ca" gracePeriod=30 Jul 06 18:30:19.569055 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:19.569029 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8c9f6f57c-52jbr" Jul 06 18:30:19.616953 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:19.616918 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rp242\" (UniqueName: \"kubernetes.io/projected/7bc983f8-7cac-4d08-baa2-5ff0816784fa-kube-api-access-rp242\") pod \"7bc983f8-7cac-4d08-baa2-5ff0816784fa\" (UID: \"7bc983f8-7cac-4d08-baa2-5ff0816784fa\") " Jul 06 18:30:19.617124 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:19.616998 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7bc983f8-7cac-4d08-baa2-5ff0816784fa-tls-cert\") pod \"7bc983f8-7cac-4d08-baa2-5ff0816784fa\" (UID: \"7bc983f8-7cac-4d08-baa2-5ff0816784fa\") " Jul 06 18:30:19.619122 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:19.619096 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bc983f8-7cac-4d08-baa2-5ff0816784fa-kube-api-access-rp242" (OuterVolumeSpecName: "kube-api-access-rp242") pod "7bc983f8-7cac-4d08-baa2-5ff0816784fa" (UID: "7bc983f8-7cac-4d08-baa2-5ff0816784fa"). InnerVolumeSpecName "kube-api-access-rp242". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:30:19.627541 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:19.627515 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bc983f8-7cac-4d08-baa2-5ff0816784fa-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "7bc983f8-7cac-4d08-baa2-5ff0816784fa" (UID: "7bc983f8-7cac-4d08-baa2-5ff0816784fa"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:30:19.718178 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:19.718146 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rp242\" (UniqueName: \"kubernetes.io/projected/7bc983f8-7cac-4d08-baa2-5ff0816784fa-kube-api-access-rp242\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:30:19.718178 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:19.718173 2566 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7bc983f8-7cac-4d08-baa2-5ff0816784fa-tls-cert\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:30:19.942179 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:19.942148 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:30:20.057395 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:20.057363 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:30:20.262740 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:20.262707 2566 generic.go:356] "Generic (PLEG): container finished" podID="7bc983f8-7cac-4d08-baa2-5ff0816784fa" containerID="36745e8c0353dceb8ca4132efa9b05ee6f2f2b87e64b5502ca7008344eda48ca" exitCode=0 Jul 06 18:30:20.263173 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:20.262756 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8c9f6f57c-52jbr" Jul 06 18:30:20.263173 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:20.262798 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8c9f6f57c-52jbr" event={"ID":"7bc983f8-7cac-4d08-baa2-5ff0816784fa","Type":"ContainerDied","Data":"36745e8c0353dceb8ca4132efa9b05ee6f2f2b87e64b5502ca7008344eda48ca"} Jul 06 18:30:20.263173 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:20.262838 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8c9f6f57c-52jbr" event={"ID":"7bc983f8-7cac-4d08-baa2-5ff0816784fa","Type":"ContainerDied","Data":"00554d7b270cdf4a9effbc15eb32f654a9746ae3a59a95e370863ed3083a2384"} Jul 06 18:30:20.263173 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:20.262859 2566 scope.go:117] "RemoveContainer" containerID="36745e8c0353dceb8ca4132efa9b05ee6f2f2b87e64b5502ca7008344eda48ca" Jul 06 18:30:20.274114 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:20.274055 2566 scope.go:117] "RemoveContainer" containerID="36745e8c0353dceb8ca4132efa9b05ee6f2f2b87e64b5502ca7008344eda48ca" Jul 06 18:30:20.274728 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:30:20.274692 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36745e8c0353dceb8ca4132efa9b05ee6f2f2b87e64b5502ca7008344eda48ca\": container with ID starting with 36745e8c0353dceb8ca4132efa9b05ee6f2f2b87e64b5502ca7008344eda48ca not found: ID does not exist" containerID="36745e8c0353dceb8ca4132efa9b05ee6f2f2b87e64b5502ca7008344eda48ca" Jul 06 18:30:20.274839 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:20.274734 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36745e8c0353dceb8ca4132efa9b05ee6f2f2b87e64b5502ca7008344eda48ca"} err="failed to get container status \"36745e8c0353dceb8ca4132efa9b05ee6f2f2b87e64b5502ca7008344eda48ca\": rpc error: code = NotFound desc = could not find container \"36745e8c0353dceb8ca4132efa9b05ee6f2f2b87e64b5502ca7008344eda48ca\": container with ID starting with 36745e8c0353dceb8ca4132efa9b05ee6f2f2b87e64b5502ca7008344eda48ca not found: ID does not exist" Jul 06 18:30:20.293628 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:20.293599 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8c9f6f57c-52jbr"] Jul 06 18:30:20.295909 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:20.295881 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8c9f6f57c-52jbr"] Jul 06 18:30:21.849447 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:21.849405 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bc983f8-7cac-4d08-baa2-5ff0816784fa" path="/var/lib/kubelet/pods/7bc983f8-7cac-4d08-baa2-5ff0816784fa/volumes" Jul 06 18:30:23.505174 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:23.505112 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:30:30.277373 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:30.277341 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:30:33.839465 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:33.839428 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb"] Jul 06 18:30:33.840041 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:33.840018 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bc983f8-7cac-4d08-baa2-5ff0816784fa" containerName="authorino" Jul 06 18:30:33.840137 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:33.840043 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bc983f8-7cac-4d08-baa2-5ff0816784fa" containerName="authorino" Jul 06 18:30:33.840247 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:33.840150 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bc983f8-7cac-4d08-baa2-5ff0816784fa" containerName="authorino" Jul 06 18:30:33.846742 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:33.846712 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb" Jul 06 18:30:33.854980 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:33.853945 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 06 18:30:33.854980 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:33.853975 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-86kpz\"" Jul 06 18:30:33.854980 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:33.853945 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 18:30:33.854980 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:33.853945 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 18:30:33.855775 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:33.855756 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb"] Jul 06 18:30:33.868277 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:33.868238 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:30:33.953526 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:33.953489 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/000ee753-6ecf-4054-9d24-a849f5d9c555-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nwkb\" (UID: \"000ee753-6ecf-4054-9d24-a849f5d9c555\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb" Jul 06 18:30:33.953710 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:33.953556 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/000ee753-6ecf-4054-9d24-a849f5d9c555-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nwkb\" (UID: \"000ee753-6ecf-4054-9d24-a849f5d9c555\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb" Jul 06 18:30:33.953710 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:33.953627 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/000ee753-6ecf-4054-9d24-a849f5d9c555-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nwkb\" (UID: \"000ee753-6ecf-4054-9d24-a849f5d9c555\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb" Jul 06 18:30:33.953710 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:33.953663 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/000ee753-6ecf-4054-9d24-a849f5d9c555-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nwkb\" (UID: \"000ee753-6ecf-4054-9d24-a849f5d9c555\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb" Jul 06 18:30:33.953710 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:33.953701 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/000ee753-6ecf-4054-9d24-a849f5d9c555-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nwkb\" (UID: \"000ee753-6ecf-4054-9d24-a849f5d9c555\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb" Jul 06 18:30:33.953901 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:33.953731 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwsjm\" (UniqueName: \"kubernetes.io/projected/000ee753-6ecf-4054-9d24-a849f5d9c555-kube-api-access-wwsjm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nwkb\" (UID: \"000ee753-6ecf-4054-9d24-a849f5d9c555\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb" Jul 06 18:30:34.054855 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:34.054799 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/000ee753-6ecf-4054-9d24-a849f5d9c555-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nwkb\" (UID: \"000ee753-6ecf-4054-9d24-a849f5d9c555\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb" Jul 06 18:30:34.055087 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:34.054941 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/000ee753-6ecf-4054-9d24-a849f5d9c555-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nwkb\" (UID: \"000ee753-6ecf-4054-9d24-a849f5d9c555\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb" Jul 06 18:30:34.055087 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:34.054979 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/000ee753-6ecf-4054-9d24-a849f5d9c555-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nwkb\" (UID: \"000ee753-6ecf-4054-9d24-a849f5d9c555\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb" Jul 06 18:30:34.055087 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:34.055012 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/000ee753-6ecf-4054-9d24-a849f5d9c555-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nwkb\" (UID: \"000ee753-6ecf-4054-9d24-a849f5d9c555\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb" Jul 06 18:30:34.055087 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:34.055064 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/000ee753-6ecf-4054-9d24-a849f5d9c555-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nwkb\" (UID: \"000ee753-6ecf-4054-9d24-a849f5d9c555\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb" Jul 06 18:30:34.055291 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:34.055092 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwsjm\" (UniqueName: \"kubernetes.io/projected/000ee753-6ecf-4054-9d24-a849f5d9c555-kube-api-access-wwsjm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nwkb\" (UID: \"000ee753-6ecf-4054-9d24-a849f5d9c555\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb" Jul 06 18:30:34.055291 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:34.055206 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/000ee753-6ecf-4054-9d24-a849f5d9c555-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nwkb\" (UID: \"000ee753-6ecf-4054-9d24-a849f5d9c555\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb" Jul 06 18:30:34.055392 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:34.055353 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/000ee753-6ecf-4054-9d24-a849f5d9c555-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nwkb\" (UID: \"000ee753-6ecf-4054-9d24-a849f5d9c555\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb" Jul 06 18:30:34.055444 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:34.055424 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/000ee753-6ecf-4054-9d24-a849f5d9c555-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nwkb\" (UID: \"000ee753-6ecf-4054-9d24-a849f5d9c555\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb" Jul 06 18:30:34.057302 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:34.057274 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/000ee753-6ecf-4054-9d24-a849f5d9c555-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nwkb\" (UID: \"000ee753-6ecf-4054-9d24-a849f5d9c555\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb" Jul 06 18:30:34.057555 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:34.057535 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/000ee753-6ecf-4054-9d24-a849f5d9c555-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nwkb\" (UID: \"000ee753-6ecf-4054-9d24-a849f5d9c555\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb" Jul 06 18:30:34.063628 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:34.063599 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwsjm\" (UniqueName: \"kubernetes.io/projected/000ee753-6ecf-4054-9d24-a849f5d9c555-kube-api-access-wwsjm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-4nwkb\" (UID: \"000ee753-6ecf-4054-9d24-a849f5d9c555\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb" Jul 06 18:30:34.159691 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:34.159639 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb" Jul 06 18:30:34.300180 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:34.300151 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb"] Jul 06 18:30:34.301182 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:30:34.301156 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod000ee753_6ecf_4054_9d24_a849f5d9c555.slice/crio-07b8daa4ac2eff18dadea9867fc655f861e6b23a912cd4f734bf235cf30acf16 WatchSource:0}: Error finding container 07b8daa4ac2eff18dadea9867fc655f861e6b23a912cd4f734bf235cf30acf16: Status 404 returned error can't find the container with id 07b8daa4ac2eff18dadea9867fc655f861e6b23a912cd4f734bf235cf30acf16 Jul 06 18:30:34.349253 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:34.349223 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb" event={"ID":"000ee753-6ecf-4054-9d24-a849f5d9c555","Type":"ContainerStarted","Data":"07b8daa4ac2eff18dadea9867fc655f861e6b23a912cd4f734bf235cf30acf16"} Jul 06 18:30:36.362051 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:36.362011 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb" event={"ID":"000ee753-6ecf-4054-9d24-a849f5d9c555","Type":"ContainerStarted","Data":"29085541221fdd02221074d0fa551462ac2e3eef17c37dd657f8c66c09dd2d0b"} Jul 06 18:30:36.382770 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:36.382715 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb" podStartSLOduration=2.318233946 podStartE2EDuration="3.382697986s" totalImagesPullingTime="1.06446404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:30:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:30:36.380487755 +0000 UTC m=+653.106916714" watchObservedRunningTime="2026-07-06 18:30:36.382697986 +0000 UTC m=+653.109126943" Jul 06 18:30:44.160691 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:44.160658 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb" Jul 06 18:30:44.161112 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:44.160728 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb" Jul 06 18:30:44.173652 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:44.173627 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb" Jul 06 18:30:44.420398 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:44.420313 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-4nwkb" Jul 06 18:30:51.037754 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:51.037720 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:30:55.851973 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:55.851943 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:30:59.062177 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:30:59.062138 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:31:01.036476 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:31:01.036438 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:31:01.093274 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:31:01.093239 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:31:13.574557 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:31:13.574515 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:32:04.296226 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:04.296193 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:32:07.154231 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:07.154193 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56fc69f68d-lf7dv"] Jul 06 18:32:07.157727 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:07.157708 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56fc69f68d-lf7dv" Jul 06 18:32:07.166378 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:07.166356 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56fc69f68d-lf7dv"] Jul 06 18:32:07.306591 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:07.306543 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npjvk\" (UniqueName: \"kubernetes.io/projected/48f39331-78fc-4928-b2d4-c17c1ed14760-kube-api-access-npjvk\") pod \"authorino-56fc69f68d-lf7dv\" (UID: \"48f39331-78fc-4928-b2d4-c17c1ed14760\") " pod="kuadrant-system/authorino-56fc69f68d-lf7dv" Jul 06 18:32:07.306770 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:07.306609 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/48f39331-78fc-4928-b2d4-c17c1ed14760-oidc-ca\") pod \"authorino-56fc69f68d-lf7dv\" (UID: \"48f39331-78fc-4928-b2d4-c17c1ed14760\") " pod="kuadrant-system/authorino-56fc69f68d-lf7dv" Jul 06 18:32:07.306770 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:07.306704 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/48f39331-78fc-4928-b2d4-c17c1ed14760-tls-cert\") pod \"authorino-56fc69f68d-lf7dv\" (UID: \"48f39331-78fc-4928-b2d4-c17c1ed14760\") " pod="kuadrant-system/authorino-56fc69f68d-lf7dv" Jul 06 18:32:07.408262 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:07.408160 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npjvk\" (UniqueName: \"kubernetes.io/projected/48f39331-78fc-4928-b2d4-c17c1ed14760-kube-api-access-npjvk\") pod \"authorino-56fc69f68d-lf7dv\" (UID: \"48f39331-78fc-4928-b2d4-c17c1ed14760\") " pod="kuadrant-system/authorino-56fc69f68d-lf7dv" Jul 06 18:32:07.408262 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:07.408229 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/48f39331-78fc-4928-b2d4-c17c1ed14760-oidc-ca\") pod \"authorino-56fc69f68d-lf7dv\" (UID: \"48f39331-78fc-4928-b2d4-c17c1ed14760\") " pod="kuadrant-system/authorino-56fc69f68d-lf7dv" Jul 06 18:32:07.408262 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:07.408260 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/48f39331-78fc-4928-b2d4-c17c1ed14760-tls-cert\") pod \"authorino-56fc69f68d-lf7dv\" (UID: \"48f39331-78fc-4928-b2d4-c17c1ed14760\") " pod="kuadrant-system/authorino-56fc69f68d-lf7dv" Jul 06 18:32:07.409024 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:07.408996 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/48f39331-78fc-4928-b2d4-c17c1ed14760-oidc-ca\") pod \"authorino-56fc69f68d-lf7dv\" (UID: \"48f39331-78fc-4928-b2d4-c17c1ed14760\") " pod="kuadrant-system/authorino-56fc69f68d-lf7dv" Jul 06 18:32:07.410818 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:07.410790 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/48f39331-78fc-4928-b2d4-c17c1ed14760-tls-cert\") pod \"authorino-56fc69f68d-lf7dv\" (UID: \"48f39331-78fc-4928-b2d4-c17c1ed14760\") " pod="kuadrant-system/authorino-56fc69f68d-lf7dv" Jul 06 18:32:07.416986 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:07.416960 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-npjvk\" (UniqueName: \"kubernetes.io/projected/48f39331-78fc-4928-b2d4-c17c1ed14760-kube-api-access-npjvk\") pod \"authorino-56fc69f68d-lf7dv\" (UID: \"48f39331-78fc-4928-b2d4-c17c1ed14760\") " pod="kuadrant-system/authorino-56fc69f68d-lf7dv" Jul 06 18:32:07.468184 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:07.468149 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56fc69f68d-lf7dv" Jul 06 18:32:07.606105 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:07.606076 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56fc69f68d-lf7dv"] Jul 06 18:32:07.607504 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:32:07.607475 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48f39331_78fc_4928_b2d4_c17c1ed14760.slice/crio-9c87366cbe736fd1acd0c8cea7e5dcca47a7301dffbd4a222ec8a17b75b9d027 WatchSource:0}: Error finding container 9c87366cbe736fd1acd0c8cea7e5dcca47a7301dffbd4a222ec8a17b75b9d027: Status 404 returned error can't find the container with id 9c87366cbe736fd1acd0c8cea7e5dcca47a7301dffbd4a222ec8a17b75b9d027 Jul 06 18:32:07.879583 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:07.879490 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56fc69f68d-lf7dv" event={"ID":"48f39331-78fc-4928-b2d4-c17c1ed14760","Type":"ContainerStarted","Data":"9c87366cbe736fd1acd0c8cea7e5dcca47a7301dffbd4a222ec8a17b75b9d027"} Jul 06 18:32:08.721708 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:08.721675 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:32:08.823608 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:08.823572 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:32:08.888468 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:08.888431 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56fc69f68d-lf7dv" event={"ID":"48f39331-78fc-4928-b2d4-c17c1ed14760","Type":"ContainerStarted","Data":"0ad4a08439bbae3971465599300d163a802fc692c6cbd4223a8d343be25a559e"} Jul 06 18:32:08.909104 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:08.909050 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-56fc69f68d-lf7dv" podStartSLOduration=1.506544464 podStartE2EDuration="1.909035425s" totalImagesPullingTime="402.490961ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:32:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:32:08.906055603 +0000 UTC m=+745.632484561" watchObservedRunningTime="2026-07-06 18:32:08.909035425 +0000 UTC m=+745.635464382" Jul 06 18:32:08.964006 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:08.963967 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f9f7b785-p7x7q"] Jul 06 18:32:08.964217 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:08.964193 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-6f9f7b785-p7x7q" podUID="6ecbccc2-d982-4e9c-b6ee-2c04f32ae723" containerName="authorino" containerID="cri-o://df789e12966eee0fe78c4364081a5b924ef0a8b0446644f9a7e9253ebc9874f1" gracePeriod=30 Jul 06 18:32:09.244156 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:09.244087 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f9f7b785-p7x7q" Jul 06 18:32:09.428065 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:09.428027 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6ecbccc2-d982-4e9c-b6ee-2c04f32ae723-tls-cert\") pod \"6ecbccc2-d982-4e9c-b6ee-2c04f32ae723\" (UID: \"6ecbccc2-d982-4e9c-b6ee-2c04f32ae723\") " Jul 06 18:32:09.428326 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:09.428091 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hssw7\" (UniqueName: \"kubernetes.io/projected/6ecbccc2-d982-4e9c-b6ee-2c04f32ae723-kube-api-access-hssw7\") pod \"6ecbccc2-d982-4e9c-b6ee-2c04f32ae723\" (UID: \"6ecbccc2-d982-4e9c-b6ee-2c04f32ae723\") " Jul 06 18:32:09.428326 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:09.428142 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/6ecbccc2-d982-4e9c-b6ee-2c04f32ae723-oidc-ca\") pod \"6ecbccc2-d982-4e9c-b6ee-2c04f32ae723\" (UID: \"6ecbccc2-d982-4e9c-b6ee-2c04f32ae723\") " Jul 06 18:32:09.430503 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:09.430458 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ecbccc2-d982-4e9c-b6ee-2c04f32ae723-kube-api-access-hssw7" (OuterVolumeSpecName: "kube-api-access-hssw7") pod "6ecbccc2-d982-4e9c-b6ee-2c04f32ae723" (UID: "6ecbccc2-d982-4e9c-b6ee-2c04f32ae723"). InnerVolumeSpecName "kube-api-access-hssw7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:32:09.433415 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:09.433385 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ecbccc2-d982-4e9c-b6ee-2c04f32ae723-oidc-ca" (OuterVolumeSpecName: "oidc-ca") pod "6ecbccc2-d982-4e9c-b6ee-2c04f32ae723" (UID: "6ecbccc2-d982-4e9c-b6ee-2c04f32ae723"). InnerVolumeSpecName "oidc-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:32:09.441028 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:09.440999 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ecbccc2-d982-4e9c-b6ee-2c04f32ae723-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "6ecbccc2-d982-4e9c-b6ee-2c04f32ae723" (UID: "6ecbccc2-d982-4e9c-b6ee-2c04f32ae723"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 18:32:09.529215 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:09.529125 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hssw7\" (UniqueName: \"kubernetes.io/projected/6ecbccc2-d982-4e9c-b6ee-2c04f32ae723-kube-api-access-hssw7\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:32:09.529215 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:09.529164 2566 reconciler_common.go:299] "Volume detached for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/6ecbccc2-d982-4e9c-b6ee-2c04f32ae723-oidc-ca\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:32:09.529215 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:09.529179 2566 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6ecbccc2-d982-4e9c-b6ee-2c04f32ae723-tls-cert\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:32:09.662807 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:09.662773 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:32:09.761694 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:09.761658 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:32:09.895630 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:09.895592 2566 generic.go:356] "Generic (PLEG): container finished" podID="6ecbccc2-d982-4e9c-b6ee-2c04f32ae723" containerID="df789e12966eee0fe78c4364081a5b924ef0a8b0446644f9a7e9253ebc9874f1" exitCode=0 Jul 06 18:32:09.895838 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:09.895634 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f9f7b785-p7x7q" Jul 06 18:32:09.895838 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:09.895671 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f9f7b785-p7x7q" event={"ID":"6ecbccc2-d982-4e9c-b6ee-2c04f32ae723","Type":"ContainerDied","Data":"df789e12966eee0fe78c4364081a5b924ef0a8b0446644f9a7e9253ebc9874f1"} Jul 06 18:32:09.895838 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:09.895716 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f9f7b785-p7x7q" event={"ID":"6ecbccc2-d982-4e9c-b6ee-2c04f32ae723","Type":"ContainerDied","Data":"8a29abd83187e43201208966ac89ab189008a25f46c9cac2122ca8445d58abdd"} Jul 06 18:32:09.895838 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:09.895732 2566 scope.go:117] "RemoveContainer" containerID="df789e12966eee0fe78c4364081a5b924ef0a8b0446644f9a7e9253ebc9874f1" Jul 06 18:32:09.906384 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:09.906356 2566 scope.go:117] "RemoveContainer" containerID="df789e12966eee0fe78c4364081a5b924ef0a8b0446644f9a7e9253ebc9874f1" Jul 06 18:32:09.906775 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:32:09.906747 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df789e12966eee0fe78c4364081a5b924ef0a8b0446644f9a7e9253ebc9874f1\": container with ID starting with df789e12966eee0fe78c4364081a5b924ef0a8b0446644f9a7e9253ebc9874f1 not found: ID does not exist" containerID="df789e12966eee0fe78c4364081a5b924ef0a8b0446644f9a7e9253ebc9874f1" Jul 06 18:32:09.906898 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:09.906785 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df789e12966eee0fe78c4364081a5b924ef0a8b0446644f9a7e9253ebc9874f1"} err="failed to get container status \"df789e12966eee0fe78c4364081a5b924ef0a8b0446644f9a7e9253ebc9874f1\": rpc error: code = NotFound desc = could not find container \"df789e12966eee0fe78c4364081a5b924ef0a8b0446644f9a7e9253ebc9874f1\": container with ID starting with df789e12966eee0fe78c4364081a5b924ef0a8b0446644f9a7e9253ebc9874f1 not found: ID does not exist" Jul 06 18:32:09.917039 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:09.917010 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f9f7b785-p7x7q"] Jul 06 18:32:09.920450 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:09.920426 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6f9f7b785-p7x7q"] Jul 06 18:32:10.837800 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:10.837766 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:32:10.936448 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:10.936415 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:32:11.849831 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:32:11.849795 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ecbccc2-d982-4e9c-b6ee-2c04f32ae723" path="/var/lib/kubelet/pods/6ecbccc2-d982-4e9c-b6ee-2c04f32ae723/volumes" Jul 06 18:33:25.202798 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:33:25.202718 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:33:27.088499 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:33:27.088464 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:34:01.791279 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:34:01.791240 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:34:01.892250 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:34:01.892219 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:34:02.036605 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:34:02.036575 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:34:07.825822 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:34:07.825779 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:34:07.888243 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:34:07.888210 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:34:07.966272 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:34:07.966235 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:34:08.066257 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:34:08.066225 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:34:08.189386 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:34:08.189352 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:34:13.855174 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:34:13.855141 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:34:13.918851 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:34:13.918809 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:34:43.775452 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:34:43.775411 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffzjr_1ede441d-79f2-4365-8a08-403bf043ef73/ovn-acl-logging/0.log" Jul 06 18:34:43.777952 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:34:43.776083 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffzjr_1ede441d-79f2-4365-8a08-403bf043ef73/ovn-acl-logging/0.log" Jul 06 18:34:56.221160 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:34:56.221128 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:34:57.346006 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:34:57.345955 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:35:51.808728 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:35:51.808699 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:37:04.503454 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:37:04.503416 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:37:04.600731 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:37:04.600700 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:37:16.798502 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:37:16.798467 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:38:03.350090 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:38:03.350057 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:38:03.409162 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:38:03.409126 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:39:43.813144 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:39:43.813028 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffzjr_1ede441d-79f2-4365-8a08-403bf043ef73/ovn-acl-logging/0.log" Jul 06 18:39:43.817179 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:39:43.815266 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffzjr_1ede441d-79f2-4365-8a08-403bf043ef73/ovn-acl-logging/0.log" Jul 06 18:42:44.203120 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:42:44.203081 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:43:22.875235 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:43:22.875195 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:44:32.512461 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:44:32.512402 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:44:32.574133 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:44:32.574094 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:44:43.854488 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:44:43.854393 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffzjr_1ede441d-79f2-4365-8a08-403bf043ef73/ovn-acl-logging/0.log" Jul 06 18:44:43.858817 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:44:43.858798 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffzjr_1ede441d-79f2-4365-8a08-403bf043ef73/ovn-acl-logging/0.log" Jul 06 18:44:58.423252 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:44:58.423218 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:00.135779 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:00.135735 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29722725-fgp2l"] Jul 06 18:45:00.136198 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:00.136170 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ecbccc2-d982-4e9c-b6ee-2c04f32ae723" containerName="authorino" Jul 06 18:45:00.136198 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:00.136181 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ecbccc2-d982-4e9c-b6ee-2c04f32ae723" containerName="authorino" Jul 06 18:45:00.136272 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:00.136243 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ecbccc2-d982-4e9c-b6ee-2c04f32ae723" containerName="authorino" Jul 06 18:45:00.139271 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:00.139254 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722725-fgp2l" Jul 06 18:45:00.142243 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:00.142223 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-hgx44\"" Jul 06 18:45:00.154850 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:00.154815 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722725-fgp2l"] Jul 06 18:45:00.286621 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:00.286588 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw9ln\" (UniqueName: \"kubernetes.io/projected/0da6f7d5-8441-418d-b90c-324d1e8649dd-kube-api-access-hw9ln\") pod \"maas-api-key-cleanup-29722725-fgp2l\" (UID: \"0da6f7d5-8441-418d-b90c-324d1e8649dd\") " pod="opendatahub/maas-api-key-cleanup-29722725-fgp2l" Jul 06 18:45:00.388028 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:00.387911 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw9ln\" (UniqueName: \"kubernetes.io/projected/0da6f7d5-8441-418d-b90c-324d1e8649dd-kube-api-access-hw9ln\") pod \"maas-api-key-cleanup-29722725-fgp2l\" (UID: \"0da6f7d5-8441-418d-b90c-324d1e8649dd\") " pod="opendatahub/maas-api-key-cleanup-29722725-fgp2l" Jul 06 18:45:00.399427 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:00.399397 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw9ln\" (UniqueName: \"kubernetes.io/projected/0da6f7d5-8441-418d-b90c-324d1e8649dd-kube-api-access-hw9ln\") pod \"maas-api-key-cleanup-29722725-fgp2l\" (UID: \"0da6f7d5-8441-418d-b90c-324d1e8649dd\") " pod="opendatahub/maas-api-key-cleanup-29722725-fgp2l" Jul 06 18:45:00.449977 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:00.449945 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722725-fgp2l" Jul 06 18:45:00.587316 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:00.587285 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722725-fgp2l"] Jul 06 18:45:00.588502 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:45:00.588480 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0da6f7d5_8441_418d_b90c_324d1e8649dd.slice/crio-97619be891e0e6c0f31440f85506e4f1b76a3e58de280e14dcc55e50dcefef80 WatchSource:0}: Error finding container 97619be891e0e6c0f31440f85506e4f1b76a3e58de280e14dcc55e50dcefef80: Status 404 returned error can't find the container with id 97619be891e0e6c0f31440f85506e4f1b76a3e58de280e14dcc55e50dcefef80 Jul 06 18:45:00.590395 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:00.590380 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 18:45:01.263335 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:01.263301 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722725-fgp2l" event={"ID":"0da6f7d5-8441-418d-b90c-324d1e8649dd","Type":"ContainerStarted","Data":"97619be891e0e6c0f31440f85506e4f1b76a3e58de280e14dcc55e50dcefef80"} Jul 06 18:45:03.277854 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:03.277812 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722725-fgp2l" event={"ID":"0da6f7d5-8441-418d-b90c-324d1e8649dd","Type":"ContainerStarted","Data":"51653a08b51dea0bd9e81b31d8eb46a6754737e61d32d78d9cf0db0e34389b47"} Jul 06 18:45:03.295686 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:03.295628 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29722725-fgp2l" podStartSLOduration=0.924303427 podStartE2EDuration="3.295608863s" totalImagesPullingTime="2.371305436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:45:03.293342577 +0000 UTC m=+1520.019771535" watchObservedRunningTime="2026-07-06 18:45:03.295608863 +0000 UTC m=+1520.022037842" Jul 06 18:45:04.286712 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:04.286612 2566 generic.go:356] "Generic (PLEG): container finished" podID="0da6f7d5-8441-418d-b90c-324d1e8649dd" containerID="51653a08b51dea0bd9e81b31d8eb46a6754737e61d32d78d9cf0db0e34389b47" exitCode=0 Jul 06 18:45:04.286712 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:04.286697 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722725-fgp2l" event={"ID":"0da6f7d5-8441-418d-b90c-324d1e8649dd","Type":"ContainerDied","Data":"51653a08b51dea0bd9e81b31d8eb46a6754737e61d32d78d9cf0db0e34389b47"} Jul 06 18:45:05.433476 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:05.433447 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722725-fgp2l" Jul 06 18:45:05.541433 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:05.541397 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hw9ln\" (UniqueName: \"kubernetes.io/projected/0da6f7d5-8441-418d-b90c-324d1e8649dd-kube-api-access-hw9ln\") pod \"0da6f7d5-8441-418d-b90c-324d1e8649dd\" (UID: \"0da6f7d5-8441-418d-b90c-324d1e8649dd\") " Jul 06 18:45:05.543764 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:05.543700 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0da6f7d5-8441-418d-b90c-324d1e8649dd-kube-api-access-hw9ln" (OuterVolumeSpecName: "kube-api-access-hw9ln") pod "0da6f7d5-8441-418d-b90c-324d1e8649dd" (UID: "0da6f7d5-8441-418d-b90c-324d1e8649dd"). InnerVolumeSpecName "kube-api-access-hw9ln". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:45:05.642915 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:05.642859 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hw9ln\" (UniqueName: \"kubernetes.io/projected/0da6f7d5-8441-418d-b90c-324d1e8649dd-kube-api-access-hw9ln\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:45:05.867692 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:05.867630 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:06.299025 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:06.298996 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722725-fgp2l" Jul 06 18:45:06.299204 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:06.298998 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722725-fgp2l" event={"ID":"0da6f7d5-8441-418d-b90c-324d1e8649dd","Type":"ContainerDied","Data":"97619be891e0e6c0f31440f85506e4f1b76a3e58de280e14dcc55e50dcefef80"} Jul 06 18:45:06.299204 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:06.299117 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="97619be891e0e6c0f31440f85506e4f1b76a3e58de280e14dcc55e50dcefef80" Jul 06 18:45:14.147272 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:14.147232 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:19.265519 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:19.265480 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:22.309040 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:22.309000 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:22.402086 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:22.402046 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:22.797541 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:22.797510 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:25.889434 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:25.889399 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:25.943660 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:25.943622 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:28.652795 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:28.652761 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:28.717285 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:28.717248 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:28.826994 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:28.826964 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl"] Jul 06 18:45:28.827439 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:28.827421 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0da6f7d5-8441-418d-b90c-324d1e8649dd" containerName="cleanup" Jul 06 18:45:28.827439 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:28.827441 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0da6f7d5-8441-418d-b90c-324d1e8649dd" containerName="cleanup" Jul 06 18:45:28.827606 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:28.827576 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0da6f7d5-8441-418d-b90c-324d1e8649dd" containerName="cleanup" Jul 06 18:45:28.830908 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:28.830888 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:28.833896 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:28.833852 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-da476535-openshift-default-dockercfg-5hcbg\"" Jul 06 18:45:28.845882 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:28.845836 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl"] Jul 06 18:45:28.872222 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:28.872186 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:28.969679 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:28.969587 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:28.972791 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:28.972769 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istio-token\") pod \"e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:28.972906 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:28.972816 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-credential-socket\") pod \"e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:28.972906 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:28.972897 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-workload-certs\") pod \"e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:28.973003 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:28.972925 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istio-podinfo\") pod \"e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:28.973003 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:28.972952 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-workload-socket\") pod \"e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:28.973003 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:28.972975 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4xmg\" (UniqueName: \"kubernetes.io/projected/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-kube-api-access-q4xmg\") pod \"e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:28.973148 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:28.973007 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istiod-ca-cert\") pod \"e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:28.973148 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:28.973024 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istio-data\") pod \"e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:28.973148 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:28.973072 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istio-envoy\") pod \"e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:29.073741 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:29.073703 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istio-token\") pod \"e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:29.073940 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:29.073769 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-credential-socket\") pod \"e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:29.073940 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:29.073814 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-workload-certs\") pod \"e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:29.073940 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:29.073846 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istio-podinfo\") pod \"e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:29.073940 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:29.073916 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-workload-socket\") pod \"e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:29.074165 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:29.073953 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4xmg\" (UniqueName: \"kubernetes.io/projected/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-kube-api-access-q4xmg\") pod \"e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:29.074165 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:29.073986 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istiod-ca-cert\") pod \"e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:29.074165 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:29.074131 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istio-data\") pod \"e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:29.074329 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:29.074179 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-credential-socket\") pod \"e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:29.074329 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:29.074221 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istio-envoy\") pod \"e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:29.074329 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:29.074306 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-workload-socket\") pod \"e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:29.074476 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:29.074421 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-workload-certs\") pod \"e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:29.074476 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:29.074454 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istio-data\") pod \"e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:29.074634 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:29.074614 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istiod-ca-cert\") pod \"e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:29.076344 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:29.076320 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istio-envoy\") pod \"e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:29.076562 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:29.076546 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istio-podinfo\") pod \"e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:29.082274 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:29.082249 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4xmg\" (UniqueName: \"kubernetes.io/projected/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-kube-api-access-q4xmg\") pod \"e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:29.082482 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:29.082461 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istio-token\") pod \"e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:29.141519 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:29.141473 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:29.284956 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:29.284930 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl"] Jul 06 18:45:29.285839 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:45:29.285805 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8771cb3_ef4c_46e9_ab79_94fe0b5a3e98.slice/crio-5efbbbe7f49fda6bd26a876d9620b6b6e8501b42650944da2b9a1f2a9192de8b WatchSource:0}: Error finding container 5efbbbe7f49fda6bd26a876d9620b6b6e8501b42650944da2b9a1f2a9192de8b: Status 404 returned error can't find the container with id 5efbbbe7f49fda6bd26a876d9620b6b6e8501b42650944da2b9a1f2a9192de8b Jul 06 18:45:29.288199 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:29.288165 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:45:29.288277 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:29.288232 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:45:29.288277 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:29.288264 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:45:29.429363 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:29.429326 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" event={"ID":"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98","Type":"ContainerStarted","Data":"bd40ce8b3a113befacb2d92cf37a8aead963e8fba2ebac472fcc31679de7e511"} Jul 06 18:45:29.429553 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:29.429393 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" event={"ID":"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98","Type":"ContainerStarted","Data":"5efbbbe7f49fda6bd26a876d9620b6b6e8501b42650944da2b9a1f2a9192de8b"} Jul 06 18:45:29.457286 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:29.456962 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" podStartSLOduration=1.456939848 podStartE2EDuration="1.456939848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:45:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:45:29.452697935 +0000 UTC m=+1546.179126896" watchObservedRunningTime="2026-07-06 18:45:29.456939848 +0000 UTC m=+1546.183368810" Jul 06 18:45:30.053992 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:30.053858 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:30.141833 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:30.141799 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:30.146492 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:30.146466 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:30.149851 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:30.149826 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:30.252810 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:30.252780 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:30.435014 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:30.434981 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:30.435961 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:30.435940 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:30.611224 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:30.611194 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:30.711999 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:30.711915 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:32.365472 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:32.365441 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:35.136533 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:35.136499 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:35.300691 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:35.300653 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl"] Jul 06 18:45:35.300915 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:35.300891 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" podUID="c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98" containerName="istio-proxy" containerID="cri-o://bd40ce8b3a113befacb2d92cf37a8aead963e8fba2ebac472fcc31679de7e511" gracePeriod=30 Jul 06 18:45:35.326417 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:35.326374 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:35.477653 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:35.477616 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:35.614657 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:35.614613 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:37.223405 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:37.223370 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:40.504692 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.504657 2566 generic.go:356] "Generic (PLEG): container finished" podID="c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98" containerID="bd40ce8b3a113befacb2d92cf37a8aead963e8fba2ebac472fcc31679de7e511" exitCode=0 Jul 06 18:45:40.505143 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.504720 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" event={"ID":"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98","Type":"ContainerDied","Data":"bd40ce8b3a113befacb2d92cf37a8aead963e8fba2ebac472fcc31679de7e511"} Jul 06 18:45:40.576907 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.576820 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:40.689927 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.689848 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-workload-certs\") pod \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " Jul 06 18:45:40.689927 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.689932 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istiod-ca-cert\") pod \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " Jul 06 18:45:40.690148 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.690010 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istio-podinfo\") pod \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " Jul 06 18:45:40.690148 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.690036 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istio-token\") pod \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " Jul 06 18:45:40.690148 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.690094 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4xmg\" (UniqueName: \"kubernetes.io/projected/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-kube-api-access-q4xmg\") pod \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " Jul 06 18:45:40.690309 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.690160 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98" (UID: "c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:45:40.690309 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.690204 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-workload-socket\") pod \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " Jul 06 18:45:40.690309 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.690231 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istio-envoy\") pod \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " Jul 06 18:45:40.690309 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.690254 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istio-data\") pod \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " Jul 06 18:45:40.690309 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.690287 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-credential-socket\") pod \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\" (UID: \"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98\") " Jul 06 18:45:40.690565 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.690418 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98" (UID: "c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:45:40.690685 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.690651 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98" (UID: "c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:45:40.690885 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.690836 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98" (UID: "c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:45:40.690885 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.690692 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-workload-certs\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:45:40.691044 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.690908 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istiod-ca-cert\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:45:40.691802 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.691768 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istio-data" (OuterVolumeSpecName: "istio-data") pod "c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98" (UID: "c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:45:40.693027 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.692840 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98" (UID: "c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:45:40.693125 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.692846 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-kube-api-access-q4xmg" (OuterVolumeSpecName: "kube-api-access-q4xmg") pod "c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98" (UID: "c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98"). InnerVolumeSpecName "kube-api-access-q4xmg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:45:40.693125 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.692939 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98" (UID: "c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 18:45:40.693125 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.692989 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istio-token" (OuterVolumeSpecName: "istio-token") pod "c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98" (UID: "c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:45:40.792284 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.792247 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istio-podinfo\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:45:40.792284 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.792278 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istio-token\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:45:40.792284 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.792287 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4xmg\" (UniqueName: \"kubernetes.io/projected/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-kube-api-access-q4xmg\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:45:40.792507 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.792297 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-workload-socket\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:45:40.792507 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.792306 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istio-envoy\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:45:40.792507 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.792313 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-istio-data\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:45:40.792507 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:40.792320 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98-credential-socket\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:45:41.039086 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.039039 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:41.140140 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.140094 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:41.218060 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.218021 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455"] Jul 06 18:45:41.218907 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.218858 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98" containerName="istio-proxy" Jul 06 18:45:41.218907 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.218904 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98" containerName="istio-proxy" Jul 06 18:45:41.219107 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.219022 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98" containerName="istio-proxy" Jul 06 18:45:41.221842 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.221819 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:41.226275 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.226254 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a5159a6f-openshift-default-dockercfg-t96ms\"" Jul 06 18:45:41.234327 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.234302 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455"] Jul 06 18:45:41.265906 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.265851 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:41.362299 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.362277 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:41.398352 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.398308 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istio-token\") pod \"e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:41.398352 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.398347 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-workload-socket\") pod \"e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:41.398600 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.398382 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istio-podinfo\") pod \"e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:41.398600 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.398424 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-credential-socket\") pod \"e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:41.398600 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.398451 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istio-envoy\") pod \"e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:41.398600 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.398473 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istiod-ca-cert\") pod \"e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:41.398600 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.398500 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-workload-certs\") pod \"e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:41.398600 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.398551 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gngqd\" (UniqueName: \"kubernetes.io/projected/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-kube-api-access-gngqd\") pod \"e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:41.398600 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.398569 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istio-data\") pod \"e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:41.499547 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.499506 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istio-podinfo\") pod \"e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:41.499728 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.499565 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-credential-socket\") pod \"e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:41.499728 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.499587 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istio-envoy\") pod \"e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:41.499728 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.499606 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istiod-ca-cert\") pod \"e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:41.499728 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.499635 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-workload-certs\") pod \"e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:41.499728 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.499677 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gngqd\" (UniqueName: \"kubernetes.io/projected/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-kube-api-access-gngqd\") pod \"e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:41.499728 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.499704 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istio-data\") pod \"e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:41.500070 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.499802 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istio-token\") pod \"e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:41.500070 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.499827 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-workload-socket\") pod \"e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:41.500209 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.500182 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-workload-certs\") pod \"e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:41.500285 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.500258 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-workload-socket\") pod \"e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:41.500344 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.500312 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-credential-socket\") pod \"e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:41.500524 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.500386 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istio-data\") pod \"e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:41.500697 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.500601 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istiod-ca-cert\") pod \"e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:41.502045 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.502026 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istio-envoy\") pod \"e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:41.502143 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.502122 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istio-podinfo\") pod \"e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:41.508711 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.508689 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istio-token\") pod \"e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:41.509055 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.508741 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gngqd\" (UniqueName: \"kubernetes.io/projected/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-kube-api-access-gngqd\") pod \"e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:41.512671 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.512649 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" Jul 06 18:45:41.512671 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.512653 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl" event={"ID":"c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98","Type":"ContainerDied","Data":"5efbbbe7f49fda6bd26a876d9620b6b6e8501b42650944da2b9a1f2a9192de8b"} Jul 06 18:45:41.512910 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.512722 2566 scope.go:117] "RemoveContainer" containerID="bd40ce8b3a113befacb2d92cf37a8aead963e8fba2ebac472fcc31679de7e511" Jul 06 18:45:41.536697 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.536668 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:41.539986 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.539965 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl"] Jul 06 18:45:41.543950 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.543928 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-da476535-openshift-default-c9bb86cdf-24hbl"] Jul 06 18:45:41.679468 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.679438 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455"] Jul 06 18:45:41.681674 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:45:41.681648 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9d6111d_a70a_4b50_aa48_2a0a2bc99bc4.slice/crio-988893831b1d3e930b85d895e9412f10af4a050c484ad0edc4e75fa744c94cda WatchSource:0}: Error finding container 988893831b1d3e930b85d895e9412f10af4a050c484ad0edc4e75fa744c94cda: Status 404 returned error can't find the container with id 988893831b1d3e930b85d895e9412f10af4a050c484ad0edc4e75fa744c94cda Jul 06 18:45:41.683823 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.683790 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:45:41.683943 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.683860 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:45:41.683943 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.683929 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:45:41.853661 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:41.853619 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98" path="/var/lib/kubelet/pods/c8771cb3-ef4c-46e9-ab79-94fe0b5a3e98/volumes" Jul 06 18:45:42.521650 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:42.521619 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" event={"ID":"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4","Type":"ContainerStarted","Data":"d995c10fdef3eb5d9c8a5b18805c3a31c98699fbe976273d9a8ffda827c8b56c"} Jul 06 18:45:42.522086 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:42.521672 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" event={"ID":"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4","Type":"ContainerStarted","Data":"988893831b1d3e930b85d895e9412f10af4a050c484ad0edc4e75fa744c94cda"} Jul 06 18:45:42.537409 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:42.537370 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:42.542233 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:42.542208 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:42.549722 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:42.549695 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:42.555610 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:42.555557 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" podStartSLOduration=1.55553644 podStartE2EDuration="1.55553644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:45:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:45:42.552943683 +0000 UTC m=+1559.279372642" watchObservedRunningTime="2026-07-06 18:45:42.55553644 +0000 UTC m=+1559.281965400" Jul 06 18:45:42.644587 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:42.644555 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:42.746269 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:42.746239 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:43.042213 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:43.042168 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:43.141485 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:43.141449 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:43.242204 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:43.242175 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:43.527620 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:43.527592 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:43.528660 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:43.528637 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:44.351167 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:44.351127 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:47.679840 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:47.679805 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:47.952373 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:47.952281 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:53.940858 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:53.940822 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455"] Jul 06 18:45:53.941309 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:53.941023 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" podUID="b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4" containerName="istio-proxy" containerID="cri-o://d995c10fdef3eb5d9c8a5b18805c3a31c98699fbe976273d9a8ffda827c8b56c" gracePeriod=30 Jul 06 18:45:53.965919 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:53.965810 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:54.063634 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.063607 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:54.081359 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.081330 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:54.124000 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.123971 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istio-data\") pod \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " Jul 06 18:45:54.124000 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.124006 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istio-token\") pod \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " Jul 06 18:45:54.124235 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.124081 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-workload-socket\") pod \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " Jul 06 18:45:54.124235 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.124124 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istio-envoy\") pod \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " Jul 06 18:45:54.124235 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.124155 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-credential-socket\") pod \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " Jul 06 18:45:54.124235 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.124213 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-workload-certs\") pod \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " Jul 06 18:45:54.124449 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.124241 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istiod-ca-cert\") pod \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " Jul 06 18:45:54.124449 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.124286 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gngqd\" (UniqueName: \"kubernetes.io/projected/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-kube-api-access-gngqd\") pod \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " Jul 06 18:45:54.124449 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.124357 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istio-podinfo\") pod \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\" (UID: \"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4\") " Jul 06 18:45:54.124449 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.124436 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4" (UID: "b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:45:54.124670 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.124583 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4" (UID: "b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:45:54.124729 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.124689 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-workload-certs\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:45:54.124729 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.124705 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-workload-socket\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:45:54.124842 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.124727 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4" (UID: "b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:45:54.124932 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.124902 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istio-data" (OuterVolumeSpecName: "istio-data") pod "b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4" (UID: "b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:45:54.125188 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.125031 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4" (UID: "b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:45:54.126897 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.126848 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4" (UID: "b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:45:54.127053 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.127027 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istio-token" (OuterVolumeSpecName: "istio-token") pod "b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4" (UID: "b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:45:54.127269 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.127243 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-kube-api-access-gngqd" (OuterVolumeSpecName: "kube-api-access-gngqd") pod "b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4" (UID: "b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4"). InnerVolumeSpecName "kube-api-access-gngqd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:45:54.127364 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.127329 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4" (UID: "b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 18:45:54.226321 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.226232 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istio-data\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:45:54.226321 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.226260 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istio-token\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:45:54.226321 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.226270 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istio-envoy\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:45:54.226321 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.226279 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-credential-socket\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:45:54.226321 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.226288 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istiod-ca-cert\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:45:54.226321 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.226298 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gngqd\" (UniqueName: \"kubernetes.io/projected/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-kube-api-access-gngqd\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:45:54.226321 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.226306 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4-istio-podinfo\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:45:54.593779 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.593682 2566 generic.go:356] "Generic (PLEG): container finished" podID="b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4" containerID="d995c10fdef3eb5d9c8a5b18805c3a31c98699fbe976273d9a8ffda827c8b56c" exitCode=0 Jul 06 18:45:54.593779 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.593739 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" Jul 06 18:45:54.593779 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.593765 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" event={"ID":"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4","Type":"ContainerDied","Data":"d995c10fdef3eb5d9c8a5b18805c3a31c98699fbe976273d9a8ffda827c8b56c"} Jul 06 18:45:54.594036 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.593842 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455" event={"ID":"b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4","Type":"ContainerDied","Data":"988893831b1d3e930b85d895e9412f10af4a050c484ad0edc4e75fa744c94cda"} Jul 06 18:45:54.594036 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.593885 2566 scope.go:117] "RemoveContainer" containerID="d995c10fdef3eb5d9c8a5b18805c3a31c98699fbe976273d9a8ffda827c8b56c" Jul 06 18:45:54.604605 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.604588 2566 scope.go:117] "RemoveContainer" containerID="d995c10fdef3eb5d9c8a5b18805c3a31c98699fbe976273d9a8ffda827c8b56c" Jul 06 18:45:54.604904 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:45:54.604884 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d995c10fdef3eb5d9c8a5b18805c3a31c98699fbe976273d9a8ffda827c8b56c\": container with ID starting with d995c10fdef3eb5d9c8a5b18805c3a31c98699fbe976273d9a8ffda827c8b56c not found: ID does not exist" containerID="d995c10fdef3eb5d9c8a5b18805c3a31c98699fbe976273d9a8ffda827c8b56c" Jul 06 18:45:54.604963 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.604914 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d995c10fdef3eb5d9c8a5b18805c3a31c98699fbe976273d9a8ffda827c8b56c"} err="failed to get container status \"d995c10fdef3eb5d9c8a5b18805c3a31c98699fbe976273d9a8ffda827c8b56c\": rpc error: code = NotFound desc = could not find container \"d995c10fdef3eb5d9c8a5b18805c3a31c98699fbe976273d9a8ffda827c8b56c\": container with ID starting with d995c10fdef3eb5d9c8a5b18805c3a31c98699fbe976273d9a8ffda827c8b56c not found: ID does not exist" Jul 06 18:45:54.617018 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.616986 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455"] Jul 06 18:45:54.620764 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:54.620735 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a5159a6f-openshift-default-65d7456cbc-kn455"] Jul 06 18:45:55.580548 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:55.580521 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:45:55.850229 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:55.850151 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4" path="/var/lib/kubelet/pods/b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4/volumes" Jul 06 18:45:59.908776 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:45:59.908738 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:00.014467 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.014431 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:00.100893 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.098154 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx"] Jul 06 18:46:00.100893 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.098813 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4" containerName="istio-proxy" Jul 06 18:46:00.100893 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.098832 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4" containerName="istio-proxy" Jul 06 18:46:00.100893 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.098941 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9d6111d-a70a-4b50-aa48-2a0a2bc99bc4" containerName="istio-proxy" Jul 06 18:46:00.102621 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.102595 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:00.105755 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.105726 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-173ca26a-openshift-default-dockercfg-58hhf\"" Jul 06 18:46:00.106043 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.106018 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx"] Jul 06 18:46:00.116794 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.116768 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:00.177443 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.177342 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:00.183047 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.183027 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istio-token\") pod \"e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:00.183121 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.183082 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istio-podinfo\") pod \"e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:00.183163 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.183127 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istiod-ca-cert\") pod \"e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:00.183198 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.183163 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-workload-socket\") pod \"e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:00.183233 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.183199 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96s6m\" (UniqueName: \"kubernetes.io/projected/19aeb098-9a32-4385-9a4a-3fa069cb1de2-kube-api-access-96s6m\") pod \"e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:00.183233 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.183225 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-credential-socket\") pod \"e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:00.183302 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.183244 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istio-data\") pod \"e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:00.183337 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.183306 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istio-envoy\") pod \"e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:00.183368 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.183342 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-workload-certs\") pod \"e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:00.284262 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.284226 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istio-podinfo\") pod \"e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:00.284432 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.284278 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istiod-ca-cert\") pod \"e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:00.284432 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.284309 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-workload-socket\") pod \"e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:00.284432 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.284331 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96s6m\" (UniqueName: \"kubernetes.io/projected/19aeb098-9a32-4385-9a4a-3fa069cb1de2-kube-api-access-96s6m\") pod \"e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:00.284432 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.284359 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-credential-socket\") pod \"e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:00.284432 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.284382 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istio-data\") pod \"e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:00.284432 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.284407 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istio-envoy\") pod \"e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:00.284847 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.284442 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-workload-certs\") pod \"e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:00.284847 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.284741 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istio-token\") pod \"e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:00.284847 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.284814 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-workload-socket\") pod \"e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:00.285043 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.284897 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-credential-socket\") pod \"e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:00.285143 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.285118 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-workload-certs\") pod \"e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:00.285206 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.285130 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istiod-ca-cert\") pod \"e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:00.285206 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.285185 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istio-data\") pod \"e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:00.286822 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.286791 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istio-envoy\") pod \"e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:00.287305 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.287287 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istio-podinfo\") pod \"e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:00.293286 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.293257 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istio-token\") pod \"e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:00.293769 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.293746 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96s6m\" (UniqueName: \"kubernetes.io/projected/19aeb098-9a32-4385-9a4a-3fa069cb1de2-kube-api-access-96s6m\") pod \"e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:00.416807 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.416769 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:00.575703 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.575672 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx"] Jul 06 18:46:00.577019 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:46:00.576995 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19aeb098_9a32_4385_9a4a_3fa069cb1de2.slice/crio-091cf70ded48c1c5b9dd815cba7406fa87cfbea62c3c18eeaa57beb5adfe036f WatchSource:0}: Error finding container 091cf70ded48c1c5b9dd815cba7406fa87cfbea62c3c18eeaa57beb5adfe036f: Status 404 returned error can't find the container with id 091cf70ded48c1c5b9dd815cba7406fa87cfbea62c3c18eeaa57beb5adfe036f Jul 06 18:46:00.579611 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.579579 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:46:00.579689 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.579649 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:46:00.579689 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.579677 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:46:00.629506 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:00.629474 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" event={"ID":"19aeb098-9a32-4385-9a4a-3fa069cb1de2","Type":"ContainerStarted","Data":"091cf70ded48c1c5b9dd815cba7406fa87cfbea62c3c18eeaa57beb5adfe036f"} Jul 06 18:46:01.326111 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:01.326076 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:01.418145 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:01.418111 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:01.475033 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:01.474977 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:01.636321 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:01.636232 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" event={"ID":"19aeb098-9a32-4385-9a4a-3fa069cb1de2","Type":"ContainerStarted","Data":"ad4b5ff69c87f7c986d8ec3624b334be1b0ff7c0a20677f12b9f10c5e557cd35"} Jul 06 18:46:01.663474 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:01.663424 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" podStartSLOduration=1.6634076260000001 podStartE2EDuration="1.663407626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:46:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:46:01.660029437 +0000 UTC m=+1578.386458602" watchObservedRunningTime="2026-07-06 18:46:01.663407626 +0000 UTC m=+1578.389836583" Jul 06 18:46:01.775368 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:01.775331 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:01.991074 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:01.991041 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:02.091100 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:02.091065 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:02.417452 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:02.417408 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:02.422535 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:02.422500 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:02.642270 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:02.642242 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:02.643212 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:02.643187 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:03.125488 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:03.125457 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:05.886137 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:05.886100 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:06.027853 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:06.027815 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx"] Jul 06 18:46:06.028068 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:06.028014 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" podUID="19aeb098-9a32-4385-9a4a-3fa069cb1de2" containerName="istio-proxy" containerID="cri-o://ad4b5ff69c87f7c986d8ec3624b334be1b0ff7c0a20677f12b9f10c5e557cd35" gracePeriod=30 Jul 06 18:46:06.056064 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:06.056030 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:06.158891 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:06.158831 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:06.254678 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:06.254630 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:09.410347 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:09.410315 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:11.282293 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.282267 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:11.403125 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.403091 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-workload-socket\") pod \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " Jul 06 18:46:11.403271 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.403146 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96s6m\" (UniqueName: \"kubernetes.io/projected/19aeb098-9a32-4385-9a4a-3fa069cb1de2-kube-api-access-96s6m\") pod \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " Jul 06 18:46:11.403271 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.403225 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-workload-certs\") pod \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " Jul 06 18:46:11.403271 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.403261 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istio-data\") pod \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " Jul 06 18:46:11.403402 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.403296 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istiod-ca-cert\") pod \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " Jul 06 18:46:11.403402 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.403319 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-credential-socket\") pod \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " Jul 06 18:46:11.403402 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.403378 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "19aeb098-9a32-4385-9a4a-3fa069cb1de2" (UID: "19aeb098-9a32-4385-9a4a-3fa069cb1de2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:46:11.403402 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.403394 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istio-podinfo\") pod \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " Jul 06 18:46:11.403566 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.403482 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istio-token\") pod \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " Jul 06 18:46:11.403566 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.403516 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istio-envoy\") pod \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\" (UID: \"19aeb098-9a32-4385-9a4a-3fa069cb1de2\") " Jul 06 18:46:11.403566 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.403527 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "19aeb098-9a32-4385-9a4a-3fa069cb1de2" (UID: "19aeb098-9a32-4385-9a4a-3fa069cb1de2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:46:11.403720 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.403603 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istio-data" (OuterVolumeSpecName: "istio-data") pod "19aeb098-9a32-4385-9a4a-3fa069cb1de2" (UID: "19aeb098-9a32-4385-9a4a-3fa069cb1de2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:46:11.403779 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.403726 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "19aeb098-9a32-4385-9a4a-3fa069cb1de2" (UID: "19aeb098-9a32-4385-9a4a-3fa069cb1de2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:46:11.403779 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.403735 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "19aeb098-9a32-4385-9a4a-3fa069cb1de2" (UID: "19aeb098-9a32-4385-9a4a-3fa069cb1de2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:46:11.403949 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.403932 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-workload-socket\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:46:11.404010 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.403953 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-workload-certs\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:46:11.404010 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.403967 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istio-data\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:46:11.404010 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.403980 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istiod-ca-cert\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:46:11.404010 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.403993 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-credential-socket\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:46:11.405506 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.405478 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19aeb098-9a32-4385-9a4a-3fa069cb1de2-kube-api-access-96s6m" (OuterVolumeSpecName: "kube-api-access-96s6m") pod "19aeb098-9a32-4385-9a4a-3fa069cb1de2" (UID: "19aeb098-9a32-4385-9a4a-3fa069cb1de2"). InnerVolumeSpecName "kube-api-access-96s6m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:46:11.405651 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.405634 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "19aeb098-9a32-4385-9a4a-3fa069cb1de2" (UID: "19aeb098-9a32-4385-9a4a-3fa069cb1de2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:46:11.405700 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.405646 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istio-token" (OuterVolumeSpecName: "istio-token") pod "19aeb098-9a32-4385-9a4a-3fa069cb1de2" (UID: "19aeb098-9a32-4385-9a4a-3fa069cb1de2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:46:11.405735 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.405705 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "19aeb098-9a32-4385-9a4a-3fa069cb1de2" (UID: "19aeb098-9a32-4385-9a4a-3fa069cb1de2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 18:46:11.504638 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.504597 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istio-podinfo\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:46:11.504638 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.504628 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istio-token\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:46:11.504638 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.504636 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19aeb098-9a32-4385-9a4a-3fa069cb1de2-istio-envoy\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:46:11.504638 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.504644 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96s6m\" (UniqueName: \"kubernetes.io/projected/19aeb098-9a32-4385-9a4a-3fa069cb1de2-kube-api-access-96s6m\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:46:11.696011 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.695922 2566 generic.go:356] "Generic (PLEG): container finished" podID="19aeb098-9a32-4385-9a4a-3fa069cb1de2" containerID="ad4b5ff69c87f7c986d8ec3624b334be1b0ff7c0a20677f12b9f10c5e557cd35" exitCode=0 Jul 06 18:46:11.696011 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.695972 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" Jul 06 18:46:11.696208 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.695974 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" event={"ID":"19aeb098-9a32-4385-9a4a-3fa069cb1de2","Type":"ContainerDied","Data":"ad4b5ff69c87f7c986d8ec3624b334be1b0ff7c0a20677f12b9f10c5e557cd35"} Jul 06 18:46:11.696208 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.696090 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx" event={"ID":"19aeb098-9a32-4385-9a4a-3fa069cb1de2","Type":"ContainerDied","Data":"091cf70ded48c1c5b9dd815cba7406fa87cfbea62c3c18eeaa57beb5adfe036f"} Jul 06 18:46:11.696208 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.696119 2566 scope.go:117] "RemoveContainer" containerID="ad4b5ff69c87f7c986d8ec3624b334be1b0ff7c0a20677f12b9f10c5e557cd35" Jul 06 18:46:11.709029 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.709010 2566 scope.go:117] "RemoveContainer" containerID="ad4b5ff69c87f7c986d8ec3624b334be1b0ff7c0a20677f12b9f10c5e557cd35" Jul 06 18:46:11.709313 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:46:11.709293 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad4b5ff69c87f7c986d8ec3624b334be1b0ff7c0a20677f12b9f10c5e557cd35\": container with ID starting with ad4b5ff69c87f7c986d8ec3624b334be1b0ff7c0a20677f12b9f10c5e557cd35 not found: ID does not exist" containerID="ad4b5ff69c87f7c986d8ec3624b334be1b0ff7c0a20677f12b9f10c5e557cd35" Jul 06 18:46:11.709354 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.709324 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad4b5ff69c87f7c986d8ec3624b334be1b0ff7c0a20677f12b9f10c5e557cd35"} err="failed to get container status \"ad4b5ff69c87f7c986d8ec3624b334be1b0ff7c0a20677f12b9f10c5e557cd35\": rpc error: code = NotFound desc = could not find container \"ad4b5ff69c87f7c986d8ec3624b334be1b0ff7c0a20677f12b9f10c5e557cd35\": container with ID starting with ad4b5ff69c87f7c986d8ec3624b334be1b0ff7c0a20677f12b9f10c5e557cd35 not found: ID does not exist" Jul 06 18:46:11.723040 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.723011 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx"] Jul 06 18:46:11.729508 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.729479 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-173ca26a-openshift-default-5cbfbb4c79-z4znx"] Jul 06 18:46:11.849498 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:11.849461 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19aeb098-9a32-4385-9a4a-3fa069cb1de2" path="/var/lib/kubelet/pods/19aeb098-9a32-4385-9a4a-3fa069cb1de2/volumes" Jul 06 18:46:12.019348 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:12.019265 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:12.089715 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:12.089678 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:12.506354 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:12.506321 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:12.608972 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:12.608935 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:12.705361 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:12.705328 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:14.104516 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:14.104459 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:19.942181 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:19.942147 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:20.055463 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:20.055420 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:20.145246 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:20.145213 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:20.242402 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:20.242310 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:20.662939 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:20.662904 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:46.650533 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:46.650497 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:52.440836 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:52.440798 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:52.545264 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:52.545225 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:46:52.643623 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:46:52.643585 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:47:08.520666 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:08.520634 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:47:14.391725 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:14.391684 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:47:14.463345 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:14.463309 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:47:14.601047 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:14.601009 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:47:25.157346 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:25.157310 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:47:25.920021 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:25.919986 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:47:31.814305 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:31.814257 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:47:31.889992 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:31.889959 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:47:31.986610 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:31.986578 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:47:32.088814 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:32.088728 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:47:33.403657 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:33.403618 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:47:40.040117 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:40.040078 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:47:40.148231 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:40.148203 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:47:40.259224 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:40.259194 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:47:40.361257 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:40.361143 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:47:41.259087 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:41.259058 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:47:41.361957 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:41.361925 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:47:43.268513 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:43.268474 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:47:50.558523 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:50.558488 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:47:52.378112 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:52.378079 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:47:57.804181 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:57.804144 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:47:57.979222 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:57.979189 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:47:58.073501 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.073415 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:47:58.468281 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.468246 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9"] Jul 06 18:47:58.468638 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.468626 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19aeb098-9a32-4385-9a4a-3fa069cb1de2" containerName="istio-proxy" Jul 06 18:47:58.468686 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.468640 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="19aeb098-9a32-4385-9a4a-3fa069cb1de2" containerName="istio-proxy" Jul 06 18:47:58.468721 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.468714 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="19aeb098-9a32-4385-9a4a-3fa069cb1de2" containerName="istio-proxy" Jul 06 18:47:58.471827 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.471811 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:47:58.476050 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.476020 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-1732de50-openshift-default-dockercfg-bvr6c\"" Jul 06 18:47:58.522665 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.522625 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-workload-socket\") pod \"e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:47:58.522665 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.522665 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcr5t\" (UniqueName: \"kubernetes.io/projected/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-kube-api-access-rcr5t\") pod \"e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:47:58.522905 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.522762 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istiod-ca-cert\") pod \"e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:47:58.522905 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.522797 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istio-token\") pod \"e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:47:58.522905 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.522822 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istio-podinfo\") pod \"e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:47:58.522905 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.522851 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-workload-certs\") pod \"e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:47:58.523046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.522961 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istio-envoy\") pod \"e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:47:58.523046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.522990 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istio-data\") pod \"e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:47:58.523046 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.523006 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-credential-socket\") pod \"e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:47:58.525536 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.525505 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9"] Jul 06 18:47:58.568208 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.568172 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:47:58.624184 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.624140 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istio-envoy\") pod \"e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:47:58.624184 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.624184 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istio-data\") pod \"e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:47:58.624435 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.624204 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-credential-socket\") pod \"e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:47:58.624435 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.624233 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-workload-socket\") pod \"e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:47:58.624435 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.624249 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcr5t\" (UniqueName: \"kubernetes.io/projected/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-kube-api-access-rcr5t\") pod \"e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:47:58.624435 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.624287 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istiod-ca-cert\") pod \"e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:47:58.624435 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.624307 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istio-token\") pod \"e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:47:58.624435 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.624333 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istio-podinfo\") pod \"e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:47:58.624435 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.624390 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-workload-certs\") pod \"e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:47:58.624815 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.624599 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istio-data\") pod \"e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:47:58.624815 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.624647 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-workload-socket\") pod \"e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:47:58.624815 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.624739 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-workload-certs\") pod \"e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:47:58.624815 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.624755 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-credential-socket\") pod \"e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:47:58.625124 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.625101 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istiod-ca-cert\") pod \"e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:47:58.626662 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.626637 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istio-envoy\") pod \"e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:47:58.627160 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.627142 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istio-podinfo\") pod \"e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:47:58.640377 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.640340 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istio-token\") pod \"e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:47:58.660838 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.660799 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcr5t\" (UniqueName: \"kubernetes.io/projected/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-kube-api-access-rcr5t\") pod \"e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:47:58.782568 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.782476 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:47:58.819473 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.819435 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:47:58.943371 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.943345 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9"] Jul 06 18:47:58.945061 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:47:58.945035 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50dbd26f_9eb0_41e5_afb5_b7f7a5d465b2.slice/crio-5b351f95a11e65a1bc22b61c8bd33cc9bdbfcac049270db8b1902e77d012baf0 WatchSource:0}: Error finding container 5b351f95a11e65a1bc22b61c8bd33cc9bdbfcac049270db8b1902e77d012baf0: Status 404 returned error can't find the container with id 5b351f95a11e65a1bc22b61c8bd33cc9bdbfcac049270db8b1902e77d012baf0 Jul 06 18:47:58.947235 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.947202 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:47:58.947357 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.947284 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:47:58.947357 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:58.947337 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:47:59.311321 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:59.311277 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" event={"ID":"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2","Type":"ContainerStarted","Data":"50fa3b370c2f53812da75d93e4ae3b98a18a586e54c994b934ece505cb10118a"} Jul 06 18:47:59.311503 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:59.311343 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" event={"ID":"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2","Type":"ContainerStarted","Data":"5b351f95a11e65a1bc22b61c8bd33cc9bdbfcac049270db8b1902e77d012baf0"} Jul 06 18:47:59.337045 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:59.336973 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" podStartSLOduration=1.336952493 podStartE2EDuration="1.336952493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:47:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:47:59.335305375 +0000 UTC m=+1696.061734334" watchObservedRunningTime="2026-07-06 18:47:59.336952493 +0000 UTC m=+1696.063381452" Jul 06 18:47:59.782720 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:47:59.782681 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:48:00.788562 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:00.788537 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:48:01.331054 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:01.331009 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:48:01.332231 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:01.332208 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:48:01.588456 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:01.588362 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:01.767473 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:01.767438 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:01.899292 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:01.899254 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:02.273520 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:02.273436 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:02.501661 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:02.501628 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:02.608554 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:02.608475 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:05.995679 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:05.995602 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:06.694232 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:06.694202 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:06.868799 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:06.868768 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:07.216073 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:07.216018 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:13.380621 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:13.380588 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:13.462776 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:13.462739 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:13.871362 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:13.871331 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:14.130324 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:14.130256 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:14.232895 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:14.229722 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:14.408116 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:14.408078 2566 generic.go:356] "Generic (PLEG): container finished" podID="50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2" containerID="50fa3b370c2f53812da75d93e4ae3b98a18a586e54c994b934ece505cb10118a" exitCode=0 Jul 06 18:48:14.408496 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:14.408146 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" event={"ID":"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2","Type":"ContainerDied","Data":"50fa3b370c2f53812da75d93e4ae3b98a18a586e54c994b934ece505cb10118a"} Jul 06 18:48:14.408632 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:14.408615 2566 scope.go:117] "RemoveContainer" containerID="50fa3b370c2f53812da75d93e4ae3b98a18a586e54c994b934ece505cb10118a" Jul 06 18:48:14.409693 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:14.409663 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:48:14.409759 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:14.409732 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:48:14.409806 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:14.409765 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:48:15.422653 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:15.422610 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" event={"ID":"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2","Type":"ContainerStarted","Data":"72f5a5b53c9879a3f7793477ba708fcb90655ed8a657ce44707c34b5fd43a62d"} Jul 06 18:48:15.783500 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:15.783423 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:48:15.784529 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:15.784505 2566 patch_prober.go:28] interesting pod/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 06 18:48:15.784580 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:15.784559 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" podUID="50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 06 18:48:16.331774 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:16.331735 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:48:16.783563 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:16.783524 2566 patch_prober.go:28] interesting pod/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 06 18:48:16.783986 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:16.783578 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" podUID="50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 06 18:48:17.783652 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:17.783620 2566 patch_prober.go:28] interesting pod/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 06 18:48:17.784053 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:17.783674 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" podUID="50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 06 18:48:18.783286 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:18.783244 2566 patch_prober.go:28] interesting pod/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 06 18:48:18.783496 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:18.783307 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" podUID="50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 06 18:48:19.787645 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:19.787613 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:48:19.789095 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:19.789069 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:48:20.042645 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:20.042566 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:20.107102 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:20.107068 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:20.319738 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:20.319636 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9"] Jul 06 18:48:20.335803 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:20.335774 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:20.451664 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:20.451631 2566 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" secret="" err="secret \"e2e-mt-1732de50-openshift-default-dockercfg-bvr6c\" not found" Jul 06 18:48:20.633154 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:20.633073 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:20.794577 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:20.794542 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:20.891494 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:20.891462 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:20.994108 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:20.994073 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:21.455930 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:21.455890 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" podUID="50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2" containerName="istio-proxy" containerID="cri-o://72f5a5b53c9879a3f7793477ba708fcb90655ed8a657ce44707c34b5fd43a62d" gracePeriod=30 Jul 06 18:48:26.716463 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:26.716406 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:48:26.817029 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:26.816996 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istiod-ca-cert\") pod \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " Jul 06 18:48:26.817029 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:26.817035 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-workload-certs\") pod \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " Jul 06 18:48:26.817283 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:26.817078 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istio-envoy\") pod \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " Jul 06 18:48:26.817283 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:26.817094 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istio-data\") pod \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " Jul 06 18:48:26.817395 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:26.817271 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-credential-socket\") pod \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " Jul 06 18:48:26.817395 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:26.817359 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istio-podinfo\") pod \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " Jul 06 18:48:26.817395 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:26.817381 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2" (UID: "50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:48:26.817539 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:26.817408 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rcr5t\" (UniqueName: \"kubernetes.io/projected/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-kube-api-access-rcr5t\") pod \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " Jul 06 18:48:26.817539 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:26.817480 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2" (UID: "50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:48:26.817539 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:26.817499 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2" (UID: "50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:48:26.817701 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:26.817546 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-workload-socket\") pod \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " Jul 06 18:48:26.817701 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:26.817629 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istio-token\") pod \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\" (UID: \"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2\") " Jul 06 18:48:26.817846 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:26.817825 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istio-data" (OuterVolumeSpecName: "istio-data") pod "50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2" (UID: "50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:48:26.817984 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:26.817931 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2" (UID: "50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:48:26.818195 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:26.818178 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istiod-ca-cert\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:48:26.818259 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:26.818201 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-workload-certs\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:48:26.818259 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:26.818214 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istio-data\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:48:26.818259 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:26.818227 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-credential-socket\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:48:26.818259 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:26.818240 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-workload-socket\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:48:26.819728 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:26.819703 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2" (UID: "50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:48:26.820050 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:26.820029 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istio-token" (OuterVolumeSpecName: "istio-token") pod "50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2" (UID: "50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:48:26.820124 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:26.820059 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2" (UID: "50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 18:48:26.820124 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:26.820065 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-kube-api-access-rcr5t" (OuterVolumeSpecName: "kube-api-access-rcr5t") pod "50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2" (UID: "50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2"). InnerVolumeSpecName "kube-api-access-rcr5t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:48:26.919344 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:26.919306 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istio-token\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:48:26.919344 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:26.919338 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istio-envoy\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:48:26.919344 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:26.919347 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-istio-podinfo\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:48:26.919580 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:26.919357 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcr5t\" (UniqueName: \"kubernetes.io/projected/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2-kube-api-access-rcr5t\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:48:27.166718 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:27.166673 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:27.289329 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:27.289287 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:27.372519 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:27.372479 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:27.491308 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:27.491211 2566 generic.go:356] "Generic (PLEG): container finished" podID="50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2" containerID="72f5a5b53c9879a3f7793477ba708fcb90655ed8a657ce44707c34b5fd43a62d" exitCode=0 Jul 06 18:48:27.491308 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:27.491276 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" Jul 06 18:48:27.491526 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:27.491300 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" event={"ID":"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2","Type":"ContainerDied","Data":"72f5a5b53c9879a3f7793477ba708fcb90655ed8a657ce44707c34b5fd43a62d"} Jul 06 18:48:27.491526 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:27.491395 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9" event={"ID":"50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2","Type":"ContainerDied","Data":"5b351f95a11e65a1bc22b61c8bd33cc9bdbfcac049270db8b1902e77d012baf0"} Jul 06 18:48:27.491526 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:27.491427 2566 scope.go:117] "RemoveContainer" containerID="72f5a5b53c9879a3f7793477ba708fcb90655ed8a657ce44707c34b5fd43a62d" Jul 06 18:48:27.501728 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:27.501711 2566 scope.go:117] "RemoveContainer" containerID="50fa3b370c2f53812da75d93e4ae3b98a18a586e54c994b934ece505cb10118a" Jul 06 18:48:27.510814 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:27.510791 2566 scope.go:117] "RemoveContainer" containerID="72f5a5b53c9879a3f7793477ba708fcb90655ed8a657ce44707c34b5fd43a62d" Jul 06 18:48:27.511103 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:48:27.511083 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72f5a5b53c9879a3f7793477ba708fcb90655ed8a657ce44707c34b5fd43a62d\": container with ID starting with 72f5a5b53c9879a3f7793477ba708fcb90655ed8a657ce44707c34b5fd43a62d not found: ID does not exist" containerID="72f5a5b53c9879a3f7793477ba708fcb90655ed8a657ce44707c34b5fd43a62d" Jul 06 18:48:27.511153 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:27.511113 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72f5a5b53c9879a3f7793477ba708fcb90655ed8a657ce44707c34b5fd43a62d"} err="failed to get container status \"72f5a5b53c9879a3f7793477ba708fcb90655ed8a657ce44707c34b5fd43a62d\": rpc error: code = NotFound desc = could not find container \"72f5a5b53c9879a3f7793477ba708fcb90655ed8a657ce44707c34b5fd43a62d\": container with ID starting with 72f5a5b53c9879a3f7793477ba708fcb90655ed8a657ce44707c34b5fd43a62d not found: ID does not exist" Jul 06 18:48:27.511153 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:27.511130 2566 scope.go:117] "RemoveContainer" containerID="50fa3b370c2f53812da75d93e4ae3b98a18a586e54c994b934ece505cb10118a" Jul 06 18:48:27.511377 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:48:27.511353 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50fa3b370c2f53812da75d93e4ae3b98a18a586e54c994b934ece505cb10118a\": container with ID starting with 50fa3b370c2f53812da75d93e4ae3b98a18a586e54c994b934ece505cb10118a not found: ID does not exist" containerID="50fa3b370c2f53812da75d93e4ae3b98a18a586e54c994b934ece505cb10118a" Jul 06 18:48:27.511436 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:27.511384 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50fa3b370c2f53812da75d93e4ae3b98a18a586e54c994b934ece505cb10118a"} err="failed to get container status \"50fa3b370c2f53812da75d93e4ae3b98a18a586e54c994b934ece505cb10118a\": rpc error: code = NotFound desc = could not find container \"50fa3b370c2f53812da75d93e4ae3b98a18a586e54c994b934ece505cb10118a\": container with ID starting with 50fa3b370c2f53812da75d93e4ae3b98a18a586e54c994b934ece505cb10118a not found: ID does not exist" Jul 06 18:48:27.514996 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:27.514974 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9"] Jul 06 18:48:27.516860 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:27.516837 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:27.518899 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:27.518880 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-1732de50-openshift-default-85965ccbcf-sd5s9"] Jul 06 18:48:27.849278 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:27.849196 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2" path="/var/lib/kubelet/pods/50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2/volumes" Jul 06 18:48:31.271288 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:31.271258 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:33.361833 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:33.361796 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:33.431297 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:33.431264 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:39.865045 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:39.865009 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:40.186962 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.186925 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n"] Jul 06 18:48:40.187421 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.187404 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2" containerName="istio-proxy" Jul 06 18:48:40.187516 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.187425 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2" containerName="istio-proxy" Jul 06 18:48:40.187578 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.187514 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2" containerName="istio-proxy" Jul 06 18:48:40.187578 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.187527 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2" containerName="istio-proxy" Jul 06 18:48:40.192137 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.192114 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:40.195061 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.195042 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-f8683f4a-openshift-default-dockercfg-rnwvd\"" Jul 06 18:48:40.205747 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.205726 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n"] Jul 06 18:48:40.224767 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.224725 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:40.242648 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.242618 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-credential-socket\") pod \"e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:40.242785 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.242660 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istio-podinfo\") pod \"e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:40.242785 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.242689 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istiod-ca-cert\") pod \"e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:40.242785 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.242757 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-workload-certs\") pod \"e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:40.242916 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.242803 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbf5j\" (UniqueName: \"kubernetes.io/projected/bb367bff-1772-45e4-9df4-5e41a8c2a15a-kube-api-access-nbf5j\") pod \"e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:40.242916 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.242828 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istio-envoy\") pod \"e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:40.242916 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.242844 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istio-token\") pod \"e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:40.243015 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.242918 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-workload-socket\") pod \"e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:40.243015 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.242965 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istio-data\") pod \"e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:40.344297 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.344262 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istiod-ca-cert\") pod \"e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:40.344487 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.344312 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-workload-certs\") pod \"e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:40.344487 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.344345 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbf5j\" (UniqueName: \"kubernetes.io/projected/bb367bff-1772-45e4-9df4-5e41a8c2a15a-kube-api-access-nbf5j\") pod \"e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:40.344487 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.344370 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istio-envoy\") pod \"e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:40.344487 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.344394 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istio-token\") pod \"e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:40.344487 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.344414 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-workload-socket\") pod \"e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:40.344487 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.344463 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istio-data\") pod \"e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:40.344808 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.344528 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-credential-socket\") pod \"e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:40.344808 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.344577 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istio-podinfo\") pod \"e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:40.344947 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.344906 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-workload-socket\") pod \"e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:40.345029 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.344997 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istio-data\") pod \"e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:40.345092 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.345067 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-credential-socket\") pod \"e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:40.345148 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.345090 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-workload-certs\") pod \"e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:40.345148 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.345075 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istiod-ca-cert\") pod \"e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:40.346815 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.346794 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istio-envoy\") pod \"e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:40.347144 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.347129 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istio-podinfo\") pod \"e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:40.354589 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.354564 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istio-token\") pod \"e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:40.354742 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.354718 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbf5j\" (UniqueName: \"kubernetes.io/projected/bb367bff-1772-45e4-9df4-5e41a8c2a15a-kube-api-access-nbf5j\") pod \"e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:40.502841 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.502750 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:40.562278 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.562247 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:40.651949 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.651919 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n"] Jul 06 18:48:40.653054 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:48:40.653023 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb367bff_1772_45e4_9df4_5e41a8c2a15a.slice/crio-56042798a10c21b3704e4d1648793ce27cf869586f893405eff53bdd019e9f71 WatchSource:0}: Error finding container 56042798a10c21b3704e4d1648793ce27cf869586f893405eff53bdd019e9f71: Status 404 returned error can't find the container with id 56042798a10c21b3704e4d1648793ce27cf869586f893405eff53bdd019e9f71 Jul 06 18:48:40.655223 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.655192 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:48:40.655305 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.655255 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:48:40.655305 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:40.655284 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:48:41.581693 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:41.581657 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" event={"ID":"bb367bff-1772-45e4-9df4-5e41a8c2a15a","Type":"ContainerStarted","Data":"46c95a6fc3b929e0ceda99852d9a26eec2039dca58828b51e443b71ac57caf2b"} Jul 06 18:48:41.582105 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:41.581712 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" event={"ID":"bb367bff-1772-45e4-9df4-5e41a8c2a15a","Type":"ContainerStarted","Data":"56042798a10c21b3704e4d1648793ce27cf869586f893405eff53bdd019e9f71"} Jul 06 18:48:41.606400 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:41.606348 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" podStartSLOduration=1.606332719 podStartE2EDuration="1.606332719s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:48:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:48:41.603224855 +0000 UTC m=+1738.329653813" watchObservedRunningTime="2026-07-06 18:48:41.606332719 +0000 UTC m=+1738.332761677" Jul 06 18:48:42.503202 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:42.503168 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:42.508550 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:42.508525 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:42.587297 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:42.587264 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:42.588289 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:42.588265 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:48:43.645758 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:43.645722 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:43.780439 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:43.780397 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:44.147347 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:44.147287 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:44.245947 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:44.245847 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:48.082585 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:48.082551 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:49.304560 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:49.304519 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:49.404848 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:49.404816 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:58.402826 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:58.401643 2566 patch_prober.go:28] interesting pod/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": read tcp 10.134.0.2:38966->10.134.0.61:15021: read: connection reset by peer" start-of-body= Jul 06 18:48:58.402826 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:58.401717 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" podUID="bb367bff-1772-45e4-9df4-5e41a8c2a15a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": read tcp 10.134.0.2:38966->10.134.0.61:15021: read: connection reset by peer" Jul 06 18:48:58.683304 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:58.683226 2566 generic.go:356] "Generic (PLEG): container finished" podID="bb367bff-1772-45e4-9df4-5e41a8c2a15a" containerID="46c95a6fc3b929e0ceda99852d9a26eec2039dca58828b51e443b71ac57caf2b" exitCode=0 Jul 06 18:48:58.683304 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:58.683289 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" event={"ID":"bb367bff-1772-45e4-9df4-5e41a8c2a15a","Type":"ContainerDied","Data":"46c95a6fc3b929e0ceda99852d9a26eec2039dca58828b51e443b71ac57caf2b"} Jul 06 18:48:58.683793 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:58.683776 2566 scope.go:117] "RemoveContainer" containerID="46c95a6fc3b929e0ceda99852d9a26eec2039dca58828b51e443b71ac57caf2b" Jul 06 18:48:58.684892 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:58.684832 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:48:58.685005 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:58.684934 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:48:58.685005 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:58.684995 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:48:58.851055 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:58.851008 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:48:59.690790 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:48:59.690748 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" event={"ID":"bb367bff-1772-45e4-9df4-5e41a8c2a15a","Type":"ContainerStarted","Data":"05d29d74d5606792e9013fda46f4ab2a5a738e8f17c9b3ba5629f82d01e53b5c"} Jul 06 18:49:00.503822 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:49:00.503784 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:49:00.504140 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:49:00.504037 2566 patch_prober.go:28] interesting pod/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 06 18:49:00.504353 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:49:00.504329 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" podUID="bb367bff-1772-45e4-9df4-5e41a8c2a15a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 06 18:49:01.503764 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:49:01.503716 2566 patch_prober.go:28] interesting pod/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 06 18:49:01.504451 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:49:01.504418 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" podUID="bb367bff-1772-45e4-9df4-5e41a8c2a15a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 06 18:49:02.503806 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:49:02.503756 2566 patch_prober.go:28] interesting pod/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 06 18:49:02.504357 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:49:02.503843 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" podUID="bb367bff-1772-45e4-9df4-5e41a8c2a15a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 06 18:49:03.503688 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:49:03.503646 2566 patch_prober.go:28] interesting pod/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 06 18:49:03.504049 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:49:03.504015 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" podUID="bb367bff-1772-45e4-9df4-5e41a8c2a15a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 06 18:49:04.507339 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:49:04.507310 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:49:04.507746 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:49:04.507586 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:49:04.508386 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:49:04.508371 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:49:04.998924 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:49:04.998891 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:49:05.099134 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:49:05.099100 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:49:31.098405 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:49:31.098360 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:49:31.468604 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:49:31.468571 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:49:37.217724 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:49:37.217636 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:49:42.232260 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:49:42.232223 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:49:43.893217 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:49:43.893189 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffzjr_1ede441d-79f2-4365-8a08-403bf043ef73/ovn-acl-logging/0.log" Jul 06 18:49:43.898718 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:49:43.898692 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffzjr_1ede441d-79f2-4365-8a08-403bf043ef73/ovn-acl-logging/0.log" Jul 06 18:50:18.328172 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:18.328136 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:50:19.463367 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:19.463327 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:50:23.909420 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:23.909386 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:50:24.010555 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:24.010514 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:50:24.989667 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:24.989638 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:50:25.059794 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:25.059748 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:50:25.243556 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:25.243472 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n"] Jul 06 18:50:25.243766 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:25.243729 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" podUID="bb367bff-1772-45e4-9df4-5e41a8c2a15a" containerName="istio-proxy" containerID="cri-o://05d29d74d5606792e9013fda46f4ab2a5a738e8f17c9b3ba5629f82d01e53b5c" gracePeriod=30 Jul 06 18:50:25.272934 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:25.272903 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:50:25.511272 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:25.511163 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:50:26.597425 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:26.597395 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:50:26.695293 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:26.695244 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:50:26.801715 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:26.801685 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:50:27.535120 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:27.535084 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:50:27.634266 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:27.634233 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:50:27.734264 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:27.734228 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:50:29.139654 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:29.139606 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:50:30.534589 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:30.534562 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:50:30.623971 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:30.623940 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-credential-socket\") pod \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " Jul 06 18:50:30.623971 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:30.623976 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istio-podinfo\") pod \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " Jul 06 18:50:30.624244 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:30.623997 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-workload-certs\") pod \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " Jul 06 18:50:30.624244 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:30.624014 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istio-token\") pod \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " Jul 06 18:50:30.624244 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:30.624063 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istio-data\") pod \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " Jul 06 18:50:30.624244 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:30.624110 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbf5j\" (UniqueName: \"kubernetes.io/projected/bb367bff-1772-45e4-9df4-5e41a8c2a15a-kube-api-access-nbf5j\") pod \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " Jul 06 18:50:30.624244 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:30.624156 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istiod-ca-cert\") pod \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " Jul 06 18:50:30.624244 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:30.624197 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istio-envoy\") pod \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " Jul 06 18:50:30.624244 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:30.624226 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-workload-socket\") pod \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\" (UID: \"bb367bff-1772-45e4-9df4-5e41a8c2a15a\") " Jul 06 18:50:30.624621 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:30.624281 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bb367bff-1772-45e4-9df4-5e41a8c2a15a" (UID: "bb367bff-1772-45e4-9df4-5e41a8c2a15a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:50:30.624621 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:30.624565 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bb367bff-1772-45e4-9df4-5e41a8c2a15a" (UID: "bb367bff-1772-45e4-9df4-5e41a8c2a15a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:50:30.624621 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:30.624580 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-workload-certs\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:50:30.624782 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:30.624716 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bb367bff-1772-45e4-9df4-5e41a8c2a15a" (UID: "bb367bff-1772-45e4-9df4-5e41a8c2a15a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:50:30.625009 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:30.624983 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istio-data" (OuterVolumeSpecName: "istio-data") pod "bb367bff-1772-45e4-9df4-5e41a8c2a15a" (UID: "bb367bff-1772-45e4-9df4-5e41a8c2a15a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:50:30.625080 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:30.624990 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bb367bff-1772-45e4-9df4-5e41a8c2a15a" (UID: "bb367bff-1772-45e4-9df4-5e41a8c2a15a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:50:30.626511 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:30.626490 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istio-token" (OuterVolumeSpecName: "istio-token") pod "bb367bff-1772-45e4-9df4-5e41a8c2a15a" (UID: "bb367bff-1772-45e4-9df4-5e41a8c2a15a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:50:30.626989 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:30.626965 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bb367bff-1772-45e4-9df4-5e41a8c2a15a" (UID: "bb367bff-1772-45e4-9df4-5e41a8c2a15a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 18:50:30.627082 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:30.626965 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bb367bff-1772-45e4-9df4-5e41a8c2a15a" (UID: "bb367bff-1772-45e4-9df4-5e41a8c2a15a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:50:30.627354 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:30.627337 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb367bff-1772-45e4-9df4-5e41a8c2a15a-kube-api-access-nbf5j" (OuterVolumeSpecName: "kube-api-access-nbf5j") pod "bb367bff-1772-45e4-9df4-5e41a8c2a15a" (UID: "bb367bff-1772-45e4-9df4-5e41a8c2a15a"). InnerVolumeSpecName "kube-api-access-nbf5j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:50:30.725309 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:30.725259 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbf5j\" (UniqueName: \"kubernetes.io/projected/bb367bff-1772-45e4-9df4-5e41a8c2a15a-kube-api-access-nbf5j\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:50:30.725309 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:30.725299 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istiod-ca-cert\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:50:30.725309 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:30.725313 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istio-envoy\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:50:30.725561 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:30.725323 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-workload-socket\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:50:30.725561 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:30.725335 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-credential-socket\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:50:30.725561 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:30.725346 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istio-podinfo\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:50:30.725561 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:30.725357 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istio-token\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:50:30.725561 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:30.725367 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb367bff-1772-45e4-9df4-5e41a8c2a15a-istio-data\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:50:31.232435 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:31.232400 2566 generic.go:356] "Generic (PLEG): container finished" podID="bb367bff-1772-45e4-9df4-5e41a8c2a15a" containerID="05d29d74d5606792e9013fda46f4ab2a5a738e8f17c9b3ba5629f82d01e53b5c" exitCode=0 Jul 06 18:50:31.232623 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:31.232448 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" Jul 06 18:50:31.232623 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:31.232470 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" event={"ID":"bb367bff-1772-45e4-9df4-5e41a8c2a15a","Type":"ContainerDied","Data":"05d29d74d5606792e9013fda46f4ab2a5a738e8f17c9b3ba5629f82d01e53b5c"} Jul 06 18:50:31.232623 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:31.232516 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n" event={"ID":"bb367bff-1772-45e4-9df4-5e41a8c2a15a","Type":"ContainerDied","Data":"56042798a10c21b3704e4d1648793ce27cf869586f893405eff53bdd019e9f71"} Jul 06 18:50:31.232623 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:31.232540 2566 scope.go:117] "RemoveContainer" containerID="05d29d74d5606792e9013fda46f4ab2a5a738e8f17c9b3ba5629f82d01e53b5c" Jul 06 18:50:31.245279 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:31.245256 2566 scope.go:117] "RemoveContainer" containerID="46c95a6fc3b929e0ceda99852d9a26eec2039dca58828b51e443b71ac57caf2b" Jul 06 18:50:31.256862 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:31.256837 2566 scope.go:117] "RemoveContainer" containerID="05d29d74d5606792e9013fda46f4ab2a5a738e8f17c9b3ba5629f82d01e53b5c" Jul 06 18:50:31.257221 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:50:31.257199 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05d29d74d5606792e9013fda46f4ab2a5a738e8f17c9b3ba5629f82d01e53b5c\": container with ID starting with 05d29d74d5606792e9013fda46f4ab2a5a738e8f17c9b3ba5629f82d01e53b5c not found: ID does not exist" containerID="05d29d74d5606792e9013fda46f4ab2a5a738e8f17c9b3ba5629f82d01e53b5c" Jul 06 18:50:31.257308 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:31.257231 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05d29d74d5606792e9013fda46f4ab2a5a738e8f17c9b3ba5629f82d01e53b5c"} err="failed to get container status \"05d29d74d5606792e9013fda46f4ab2a5a738e8f17c9b3ba5629f82d01e53b5c\": rpc error: code = NotFound desc = could not find container \"05d29d74d5606792e9013fda46f4ab2a5a738e8f17c9b3ba5629f82d01e53b5c\": container with ID starting with 05d29d74d5606792e9013fda46f4ab2a5a738e8f17c9b3ba5629f82d01e53b5c not found: ID does not exist" Jul 06 18:50:31.257308 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:31.257252 2566 scope.go:117] "RemoveContainer" containerID="46c95a6fc3b929e0ceda99852d9a26eec2039dca58828b51e443b71ac57caf2b" Jul 06 18:50:31.257543 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:31.257512 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n"] Jul 06 18:50:31.257654 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:50:31.257567 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46c95a6fc3b929e0ceda99852d9a26eec2039dca58828b51e443b71ac57caf2b\": container with ID starting with 46c95a6fc3b929e0ceda99852d9a26eec2039dca58828b51e443b71ac57caf2b not found: ID does not exist" containerID="46c95a6fc3b929e0ceda99852d9a26eec2039dca58828b51e443b71ac57caf2b" Jul 06 18:50:31.257654 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:31.257617 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46c95a6fc3b929e0ceda99852d9a26eec2039dca58828b51e443b71ac57caf2b"} err="failed to get container status \"46c95a6fc3b929e0ceda99852d9a26eec2039dca58828b51e443b71ac57caf2b\": rpc error: code = NotFound desc = could not find container \"46c95a6fc3b929e0ceda99852d9a26eec2039dca58828b51e443b71ac57caf2b\": container with ID starting with 46c95a6fc3b929e0ceda99852d9a26eec2039dca58828b51e443b71ac57caf2b not found: ID does not exist" Jul 06 18:50:31.261232 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:31.261206 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-f8683f4a-openshift-default-7959bd6cff-x566n"] Jul 06 18:50:31.852445 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:31.852412 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb367bff-1772-45e4-9df4-5e41a8c2a15a" path="/var/lib/kubelet/pods/bb367bff-1772-45e4-9df4-5e41a8c2a15a/volumes" Jul 06 18:50:34.856008 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:34.855979 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:50:34.921629 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:34.921572 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:50:35.536496 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:35.536459 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:50:41.657585 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:41.657547 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:50:41.726560 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:41.726528 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:50:41.814846 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:41.814816 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:50:52.364403 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:52.364366 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:50:52.713985 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:50:52.713946 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:06.825927 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:06.825825 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:06.889792 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:06.889754 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:13.704180 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:13.704149 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:19.612983 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:19.612947 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:19.780634 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:19.780595 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:20.006673 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.006643 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:20.321140 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.321056 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7"] Jul 06 18:51:20.321466 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.321453 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2" containerName="istio-proxy" Jul 06 18:51:20.321518 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.321467 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="50dbd26f-9eb0-41e5-afb5-b7f7a5d465b2" containerName="istio-proxy" Jul 06 18:51:20.321518 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.321478 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb367bff-1772-45e4-9df4-5e41a8c2a15a" containerName="istio-proxy" Jul 06 18:51:20.321518 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.321483 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb367bff-1772-45e4-9df4-5e41a8c2a15a" containerName="istio-proxy" Jul 06 18:51:20.321624 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.321545 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb367bff-1772-45e4-9df4-5e41a8c2a15a" containerName="istio-proxy" Jul 06 18:51:20.324228 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.324213 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:20.327260 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.327242 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-f2348c-openshift-default-dockercfg-j7p28\"" Jul 06 18:51:20.336025 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.336006 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7"] Jul 06 18:51:20.345510 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.345485 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:20.395716 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.395675 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-istio-envoy\") pod \"e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:20.395963 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.395725 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-istio-data\") pod \"e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:20.395963 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.395755 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2e30913-d907-4c06-a2ab-b53e96aace80-istiod-ca-cert\") pod \"e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:20.395963 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.395832 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2e30913-d907-4c06-a2ab-b53e96aace80-istio-token\") pod \"e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:20.395963 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.395903 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-workload-certs\") pod \"e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:20.395963 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.395925 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2e30913-d907-4c06-a2ab-b53e96aace80-istio-podinfo\") pod \"e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:20.395963 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.395949 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-credential-socket\") pod \"e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:20.396232 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.395992 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-workload-socket\") pod \"e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:20.396232 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.396050 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq6pl\" (UniqueName: \"kubernetes.io/projected/c2e30913-d907-4c06-a2ab-b53e96aace80-kube-api-access-rq6pl\") pod \"e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:20.497063 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.497014 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-istio-envoy\") pod \"e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:20.497275 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.497074 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-istio-data\") pod \"e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:20.497275 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.497101 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2e30913-d907-4c06-a2ab-b53e96aace80-istiod-ca-cert\") pod \"e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:20.497275 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.497176 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2e30913-d907-4c06-a2ab-b53e96aace80-istio-token\") pod \"e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:20.497275 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.497232 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-workload-certs\") pod \"e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:20.497275 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.497253 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2e30913-d907-4c06-a2ab-b53e96aace80-istio-podinfo\") pod \"e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:20.497544 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.497278 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-credential-socket\") pod \"e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:20.497544 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.497307 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-workload-socket\") pod \"e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:20.497544 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.497349 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq6pl\" (UniqueName: \"kubernetes.io/projected/c2e30913-d907-4c06-a2ab-b53e96aace80-kube-api-access-rq6pl\") pod \"e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:20.497707 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.497548 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-istio-data\") pod \"e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:20.497707 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.497607 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-workload-certs\") pod \"e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:20.497837 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.497814 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-credential-socket\") pod \"e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:20.497926 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.497831 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-workload-socket\") pod \"e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:20.497982 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.497961 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2e30913-d907-4c06-a2ab-b53e96aace80-istiod-ca-cert\") pod \"e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:20.499463 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.499442 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-istio-envoy\") pod \"e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:20.499925 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.499903 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2e30913-d907-4c06-a2ab-b53e96aace80-istio-podinfo\") pod \"e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:20.505985 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.505953 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2e30913-d907-4c06-a2ab-b53e96aace80-istio-token\") pod \"e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:20.506272 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.506248 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq6pl\" (UniqueName: \"kubernetes.io/projected/c2e30913-d907-4c06-a2ab-b53e96aace80-kube-api-access-rq6pl\") pod \"e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:20.635850 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.635758 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:20.781656 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.781618 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7"] Jul 06 18:51:20.782645 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:51:20.782613 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2e30913_d907_4c06_a2ab_b53e96aace80.slice/crio-2425c15652a086b6e76b1d15128242b460a854e8b1010e56ed00a0eed97e27a7 WatchSource:0}: Error finding container 2425c15652a086b6e76b1d15128242b460a854e8b1010e56ed00a0eed97e27a7: Status 404 returned error can't find the container with id 2425c15652a086b6e76b1d15128242b460a854e8b1010e56ed00a0eed97e27a7 Jul 06 18:51:20.784984 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.784955 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:51:20.785052 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.785018 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:51:20.785089 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.785050 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:51:20.797790 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:20.797763 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:21.522436 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:21.522398 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" event={"ID":"c2e30913-d907-4c06-a2ab-b53e96aace80","Type":"ContainerStarted","Data":"e2558896947a416f13a03214314666f6b44528ed5966fc120dc63d3e902d4b9c"} Jul 06 18:51:21.522616 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:21.522452 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" event={"ID":"c2e30913-d907-4c06-a2ab-b53e96aace80","Type":"ContainerStarted","Data":"2425c15652a086b6e76b1d15128242b460a854e8b1010e56ed00a0eed97e27a7"} Jul 06 18:51:21.557788 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:21.557739 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" podStartSLOduration=1.557722771 podStartE2EDuration="1.557722771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:51:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:51:21.555564959 +0000 UTC m=+1898.281993920" watchObservedRunningTime="2026-07-06 18:51:21.557722771 +0000 UTC m=+1898.284151727" Jul 06 18:51:21.636954 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:21.636909 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:21.641860 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:21.641836 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:22.528180 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:22.528146 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:22.528967 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:22.528948 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:23.672947 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:23.672907 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:23.783113 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:23.783084 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:23.881291 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:23.881262 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:24.167226 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:24.167184 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:24.394527 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:24.394493 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:24.491972 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:24.491809 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:28.193221 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:28.193160 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:28.601533 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:28.601411 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:28.956587 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:28.956555 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g"] Jul 06 18:51:28.957056 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:28.957038 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb367bff-1772-45e4-9df4-5e41a8c2a15a" containerName="istio-proxy" Jul 06 18:51:28.957172 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:28.957059 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb367bff-1772-45e4-9df4-5e41a8c2a15a" containerName="istio-proxy" Jul 06 18:51:28.957172 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:28.957155 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb367bff-1772-45e4-9df4-5e41a8c2a15a" containerName="istio-proxy" Jul 06 18:51:28.960530 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:28.960508 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:28.963611 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:28.963593 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-ef5ef7-openshift-default-dockercfg-t8hh8\"" Jul 06 18:51:28.980081 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:28.980047 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g"] Jul 06 18:51:29.007237 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.007205 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:29.080618 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.080582 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9068b778-5a45-4466-951a-3d00db9c3833-istiod-ca-cert\") pod \"e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:29.080618 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.080621 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-credential-socket\") pod \"e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:29.080847 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.080637 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-istio-envoy\") pod \"e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:29.080847 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.080716 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-istio-data\") pod \"e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:29.080847 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.080758 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-workload-certs\") pod \"e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:29.080847 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.080811 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4kc6\" (UniqueName: \"kubernetes.io/projected/9068b778-5a45-4466-951a-3d00db9c3833-kube-api-access-s4kc6\") pod \"e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:29.081031 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.080891 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9068b778-5a45-4466-951a-3d00db9c3833-istio-podinfo\") pod \"e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:29.081031 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.080935 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-workload-socket\") pod \"e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:29.081031 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.080949 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9068b778-5a45-4466-951a-3d00db9c3833-istio-token\") pod \"e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:29.148463 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.148430 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:29.181850 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.181818 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-workload-certs\") pod \"e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:29.182050 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.181861 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4kc6\" (UniqueName: \"kubernetes.io/projected/9068b778-5a45-4466-951a-3d00db9c3833-kube-api-access-s4kc6\") pod \"e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:29.182050 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.181934 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9068b778-5a45-4466-951a-3d00db9c3833-istio-podinfo\") pod \"e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:29.182050 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.181978 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-workload-socket\") pod \"e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:29.182050 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.182001 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9068b778-5a45-4466-951a-3d00db9c3833-istio-token\") pod \"e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:29.182050 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.182028 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9068b778-5a45-4466-951a-3d00db9c3833-istiod-ca-cert\") pod \"e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:29.182325 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.182062 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-credential-socket\") pod \"e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:29.182325 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.182085 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-istio-envoy\") pod \"e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:29.182325 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.182136 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-istio-data\") pod \"e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:29.182325 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.182279 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-workload-certs\") pod \"e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:29.182325 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.182322 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-workload-socket\") pod \"e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:29.182584 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.182508 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-credential-socket\") pod \"e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:29.182625 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.182608 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-istio-data\") pod \"e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:29.182740 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.182720 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9068b778-5a45-4466-951a-3d00db9c3833-istiod-ca-cert\") pod \"e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:29.184440 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.184422 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-istio-envoy\") pod \"e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:29.184641 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.184625 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9068b778-5a45-4466-951a-3d00db9c3833-istio-podinfo\") pod \"e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:29.191169 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.191140 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4kc6\" (UniqueName: \"kubernetes.io/projected/9068b778-5a45-4466-951a-3d00db9c3833-kube-api-access-s4kc6\") pod \"e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:29.191702 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.191677 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9068b778-5a45-4466-951a-3d00db9c3833-istio-token\") pod \"e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:29.271192 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.271099 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:29.415913 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.415857 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g"] Jul 06 18:51:29.416433 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:51:29.416414 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9068b778_5a45_4466_951a_3d00db9c3833.slice/crio-f83d0111e7469d3e47323b4e78d111ed637245a55390b006a367c91e2219274b WatchSource:0}: Error finding container f83d0111e7469d3e47323b4e78d111ed637245a55390b006a367c91e2219274b: Status 404 returned error can't find the container with id f83d0111e7469d3e47323b4e78d111ed637245a55390b006a367c91e2219274b Jul 06 18:51:29.418555 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.418520 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:51:29.418656 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.418588 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:51:29.418656 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.418619 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:51:29.571572 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.571474 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" event={"ID":"9068b778-5a45-4466-951a-3d00db9c3833","Type":"ContainerStarted","Data":"e10bf8ba1734672c20be97cd660cb778cffe281c03b91afa1d7be4850e7dcaa1"} Jul 06 18:51:29.571572 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.571538 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" event={"ID":"9068b778-5a45-4466-951a-3d00db9c3833","Type":"ContainerStarted","Data":"f83d0111e7469d3e47323b4e78d111ed637245a55390b006a367c91e2219274b"} Jul 06 18:51:29.591135 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:29.591083 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" podStartSLOduration=1.591064496 podStartE2EDuration="1.591064496s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:51:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:51:29.590673674 +0000 UTC m=+1906.317102633" watchObservedRunningTime="2026-07-06 18:51:29.591064496 +0000 UTC m=+1906.317493515" Jul 06 18:51:30.271732 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:30.271702 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:30.276573 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:30.276545 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:30.578002 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:30.577900 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:30.578935 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:30.578899 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:31.644558 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:31.644513 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:31.734038 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:31.734005 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:31.855195 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:31.855164 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:32.026853 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:32.026762 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:32.390054 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:32.390022 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:32.494834 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:32.494803 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:36.141545 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:36.141504 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:38.569102 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:38.569056 2566 patch_prober.go:28] interesting pod/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 18:51:38.569490 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:38.569133 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" podUID="c2e30913-d907-4c06-a2ab-b53e96aace80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 18:51:40.647593 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:40.647555 2566 generic.go:356] "Generic (PLEG): container finished" podID="c2e30913-d907-4c06-a2ab-b53e96aace80" containerID="e2558896947a416f13a03214314666f6b44528ed5966fc120dc63d3e902d4b9c" exitCode=0 Jul 06 18:51:40.648054 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:40.647630 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" event={"ID":"c2e30913-d907-4c06-a2ab-b53e96aace80","Type":"ContainerDied","Data":"e2558896947a416f13a03214314666f6b44528ed5966fc120dc63d3e902d4b9c"} Jul 06 18:51:40.648101 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:40.648067 2566 scope.go:117] "RemoveContainer" containerID="e2558896947a416f13a03214314666f6b44528ed5966fc120dc63d3e902d4b9c" Jul 06 18:51:40.649078 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:40.649045 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:51:40.649198 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:40.649131 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:51:40.649198 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:40.649179 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:51:41.656522 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:41.656488 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" event={"ID":"c2e30913-d907-4c06-a2ab-b53e96aace80","Type":"ContainerStarted","Data":"39a6e6c5a60595b89a02d1985594167ddde9b29cf25a871ec3309b0c6d244538"} Jul 06 18:51:42.637060 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:42.636544 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:42.637060 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:42.636680 2566 patch_prober.go:28] interesting pod/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 06 18:51:42.637060 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:42.636726 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" podUID="c2e30913-d907-4c06-a2ab-b53e96aace80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 06 18:51:43.637091 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:43.637048 2566 patch_prober.go:28] interesting pod/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 06 18:51:43.637522 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:43.637114 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" podUID="c2e30913-d907-4c06-a2ab-b53e96aace80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 06 18:51:44.636892 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:44.636831 2566 patch_prober.go:28] interesting pod/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 06 18:51:44.637187 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:44.636915 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" podUID="c2e30913-d907-4c06-a2ab-b53e96aace80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 06 18:51:45.280898 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:45.280838 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:45.383437 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:45.383406 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:45.636651 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:45.636567 2566 patch_prober.go:28] interesting pod/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 06 18:51:45.636651 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:45.636625 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" podUID="c2e30913-d907-4c06-a2ab-b53e96aace80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 06 18:51:46.011542 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:46.010817 2566 patch_prober.go:28] interesting pod/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": read tcp 10.134.0.2:56894->10.134.0.63:15021: read: connection reset by peer" start-of-body= Jul 06 18:51:46.011542 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:46.010901 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" podUID="9068b778-5a45-4466-951a-3d00db9c3833" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": read tcp 10.134.0.2:56894->10.134.0.63:15021: read: connection reset by peer" Jul 06 18:51:46.639946 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:46.639913 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:46.640169 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:46.640148 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:46.640933 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:46.640913 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:51:46.687737 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:46.687679 2566 generic.go:356] "Generic (PLEG): container finished" podID="9068b778-5a45-4466-951a-3d00db9c3833" containerID="e10bf8ba1734672c20be97cd660cb778cffe281c03b91afa1d7be4850e7dcaa1" exitCode=0 Jul 06 18:51:46.687966 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:46.687749 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" event={"ID":"9068b778-5a45-4466-951a-3d00db9c3833","Type":"ContainerDied","Data":"e10bf8ba1734672c20be97cd660cb778cffe281c03b91afa1d7be4850e7dcaa1"} Jul 06 18:51:46.691905 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:46.688917 2566 scope.go:117] "RemoveContainer" containerID="e10bf8ba1734672c20be97cd660cb778cffe281c03b91afa1d7be4850e7dcaa1" Jul 06 18:51:46.692434 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:46.692128 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:51:46.692505 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:46.692456 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:51:46.692703 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:46.692671 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:51:47.696161 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:47.696117 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" event={"ID":"9068b778-5a45-4466-951a-3d00db9c3833","Type":"ContainerStarted","Data":"b202355d7381f16468a8781dd3aa30b88e53ec4fc0f7e9ecabda8e7eebb39c3f"} Jul 06 18:51:48.271556 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:48.271518 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:48.271754 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:48.271732 2566 patch_prober.go:28] interesting pod/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 06 18:51:48.271803 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:48.271788 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" podUID="9068b778-5a45-4466-951a-3d00db9c3833" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 06 18:51:49.272165 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:49.272126 2566 patch_prober.go:28] interesting pod/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 06 18:51:49.272631 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:49.272197 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" podUID="9068b778-5a45-4466-951a-3d00db9c3833" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 06 18:51:50.271687 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:50.271649 2566 patch_prober.go:28] interesting pod/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 06 18:51:50.271896 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:50.271703 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" podUID="9068b778-5a45-4466-951a-3d00db9c3833" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 06 18:51:51.271753 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:51.271712 2566 patch_prober.go:28] interesting pod/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 06 18:51:51.272182 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:51.271773 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" podUID="9068b778-5a45-4466-951a-3d00db9c3833" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 06 18:51:52.050421 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:52.050387 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:52.275561 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:52.275528 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:52.275956 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:52.275909 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:52.276643 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:52.276626 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:51:53.221742 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:53.221694 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:51:53.320253 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:51:53.320210 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:52:21.385564 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:52:21.385528 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:52:55.658394 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:52:55.658353 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:52:57.289136 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:52:57.289099 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:52:57.498760 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:52:57.498724 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:52:57.672527 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:52:57.672495 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:52:57.865788 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:52:57.865750 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:53:02.302955 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:02.302920 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:53:39.605085 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:39.605031 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:53:39.919151 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:39.919120 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7"] Jul 06 18:53:39.919333 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:39.919313 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" podUID="c2e30913-d907-4c06-a2ab-b53e96aace80" containerName="istio-proxy" containerID="cri-o://39a6e6c5a60595b89a02d1985594167ddde9b29cf25a871ec3309b0c6d244538" gracePeriod=30 Jul 06 18:53:39.937480 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:39.937447 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:53:40.170145 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:40.170062 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:53:40.568935 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:40.568836 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:53:40.943577 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:40.943548 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:53:41.046956 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:41.046922 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:53:41.144718 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:41.144682 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:53:41.288492 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:41.288397 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:53:45.176567 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.176542 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:53:45.275249 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.275153 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-workload-certs\") pod \"c2e30913-d907-4c06-a2ab-b53e96aace80\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " Jul 06 18:53:45.275249 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.275200 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2e30913-d907-4c06-a2ab-b53e96aace80-istio-token\") pod \"c2e30913-d907-4c06-a2ab-b53e96aace80\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " Jul 06 18:53:45.275498 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.275260 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rq6pl\" (UniqueName: \"kubernetes.io/projected/c2e30913-d907-4c06-a2ab-b53e96aace80-kube-api-access-rq6pl\") pod \"c2e30913-d907-4c06-a2ab-b53e96aace80\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " Jul 06 18:53:45.275498 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.275279 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-workload-socket\") pod \"c2e30913-d907-4c06-a2ab-b53e96aace80\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " Jul 06 18:53:45.275498 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.275307 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2e30913-d907-4c06-a2ab-b53e96aace80-istiod-ca-cert\") pod \"c2e30913-d907-4c06-a2ab-b53e96aace80\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " Jul 06 18:53:45.275662 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.275488 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-credential-socket\") pod \"c2e30913-d907-4c06-a2ab-b53e96aace80\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " Jul 06 18:53:45.275662 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.275536 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-istio-data\") pod \"c2e30913-d907-4c06-a2ab-b53e96aace80\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " Jul 06 18:53:45.275662 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.275486 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c2e30913-d907-4c06-a2ab-b53e96aace80" (UID: "c2e30913-d907-4c06-a2ab-b53e96aace80"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:53:45.275662 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.275576 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-istio-envoy\") pod \"c2e30913-d907-4c06-a2ab-b53e96aace80\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " Jul 06 18:53:45.275662 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.275610 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2e30913-d907-4c06-a2ab-b53e96aace80-istio-podinfo\") pod \"c2e30913-d907-4c06-a2ab-b53e96aace80\" (UID: \"c2e30913-d907-4c06-a2ab-b53e96aace80\") " Jul 06 18:53:45.275662 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.275608 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c2e30913-d907-4c06-a2ab-b53e96aace80" (UID: "c2e30913-d907-4c06-a2ab-b53e96aace80"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:53:45.276001 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.275699 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c2e30913-d907-4c06-a2ab-b53e96aace80" (UID: "c2e30913-d907-4c06-a2ab-b53e96aace80"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:53:45.276001 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.275730 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2e30913-d907-4c06-a2ab-b53e96aace80-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c2e30913-d907-4c06-a2ab-b53e96aace80" (UID: "c2e30913-d907-4c06-a2ab-b53e96aace80"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:53:45.276112 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.276044 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-credential-socket\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:53:45.276303 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.276185 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-workload-certs\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:53:45.276303 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.276245 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-workload-socket\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:53:45.276303 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.276258 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2e30913-d907-4c06-a2ab-b53e96aace80-istiod-ca-cert\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:53:45.276663 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.276632 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-istio-data" (OuterVolumeSpecName: "istio-data") pod "c2e30913-d907-4c06-a2ab-b53e96aace80" (UID: "c2e30913-d907-4c06-a2ab-b53e96aace80"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:53:45.277775 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.277747 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2e30913-d907-4c06-a2ab-b53e96aace80-kube-api-access-rq6pl" (OuterVolumeSpecName: "kube-api-access-rq6pl") pod "c2e30913-d907-4c06-a2ab-b53e96aace80" (UID: "c2e30913-d907-4c06-a2ab-b53e96aace80"). InnerVolumeSpecName "kube-api-access-rq6pl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:53:45.277905 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.277844 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2e30913-d907-4c06-a2ab-b53e96aace80-istio-token" (OuterVolumeSpecName: "istio-token") pod "c2e30913-d907-4c06-a2ab-b53e96aace80" (UID: "c2e30913-d907-4c06-a2ab-b53e96aace80"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:53:45.277966 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.277901 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c2e30913-d907-4c06-a2ab-b53e96aace80" (UID: "c2e30913-d907-4c06-a2ab-b53e96aace80"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:53:45.278021 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.278006 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c2e30913-d907-4c06-a2ab-b53e96aace80-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c2e30913-d907-4c06-a2ab-b53e96aace80" (UID: "c2e30913-d907-4c06-a2ab-b53e96aace80"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 18:53:45.377486 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.377444 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rq6pl\" (UniqueName: \"kubernetes.io/projected/c2e30913-d907-4c06-a2ab-b53e96aace80-kube-api-access-rq6pl\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:53:45.377486 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.377478 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-istio-data\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:53:45.377486 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.377488 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2e30913-d907-4c06-a2ab-b53e96aace80-istio-envoy\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:53:45.377724 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.377497 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2e30913-d907-4c06-a2ab-b53e96aace80-istio-podinfo\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:53:45.377724 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.377510 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2e30913-d907-4c06-a2ab-b53e96aace80-istio-token\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:53:45.385389 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.385352 2566 generic.go:356] "Generic (PLEG): container finished" podID="c2e30913-d907-4c06-a2ab-b53e96aace80" containerID="39a6e6c5a60595b89a02d1985594167ddde9b29cf25a871ec3309b0c6d244538" exitCode=0 Jul 06 18:53:45.385558 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.385404 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" Jul 06 18:53:45.385558 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.385400 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" event={"ID":"c2e30913-d907-4c06-a2ab-b53e96aace80","Type":"ContainerDied","Data":"39a6e6c5a60595b89a02d1985594167ddde9b29cf25a871ec3309b0c6d244538"} Jul 06 18:53:45.385558 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.385528 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7" event={"ID":"c2e30913-d907-4c06-a2ab-b53e96aace80","Type":"ContainerDied","Data":"2425c15652a086b6e76b1d15128242b460a854e8b1010e56ed00a0eed97e27a7"} Jul 06 18:53:45.385558 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.385558 2566 scope.go:117] "RemoveContainer" containerID="39a6e6c5a60595b89a02d1985594167ddde9b29cf25a871ec3309b0c6d244538" Jul 06 18:53:45.396329 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.396307 2566 scope.go:117] "RemoveContainer" containerID="e2558896947a416f13a03214314666f6b44528ed5966fc120dc63d3e902d4b9c" Jul 06 18:53:45.405250 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.405231 2566 scope.go:117] "RemoveContainer" containerID="39a6e6c5a60595b89a02d1985594167ddde9b29cf25a871ec3309b0c6d244538" Jul 06 18:53:45.405509 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:53:45.405487 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39a6e6c5a60595b89a02d1985594167ddde9b29cf25a871ec3309b0c6d244538\": container with ID starting with 39a6e6c5a60595b89a02d1985594167ddde9b29cf25a871ec3309b0c6d244538 not found: ID does not exist" containerID="39a6e6c5a60595b89a02d1985594167ddde9b29cf25a871ec3309b0c6d244538" Jul 06 18:53:45.405555 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.405523 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39a6e6c5a60595b89a02d1985594167ddde9b29cf25a871ec3309b0c6d244538"} err="failed to get container status \"39a6e6c5a60595b89a02d1985594167ddde9b29cf25a871ec3309b0c6d244538\": rpc error: code = NotFound desc = could not find container \"39a6e6c5a60595b89a02d1985594167ddde9b29cf25a871ec3309b0c6d244538\": container with ID starting with 39a6e6c5a60595b89a02d1985594167ddde9b29cf25a871ec3309b0c6d244538 not found: ID does not exist" Jul 06 18:53:45.405555 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.405545 2566 scope.go:117] "RemoveContainer" containerID="e2558896947a416f13a03214314666f6b44528ed5966fc120dc63d3e902d4b9c" Jul 06 18:53:45.405753 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:53:45.405738 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2558896947a416f13a03214314666f6b44528ed5966fc120dc63d3e902d4b9c\": container with ID starting with e2558896947a416f13a03214314666f6b44528ed5966fc120dc63d3e902d4b9c not found: ID does not exist" containerID="e2558896947a416f13a03214314666f6b44528ed5966fc120dc63d3e902d4b9c" Jul 06 18:53:45.405795 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.405758 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2558896947a416f13a03214314666f6b44528ed5966fc120dc63d3e902d4b9c"} err="failed to get container status \"e2558896947a416f13a03214314666f6b44528ed5966fc120dc63d3e902d4b9c\": rpc error: code = NotFound desc = could not find container \"e2558896947a416f13a03214314666f6b44528ed5966fc120dc63d3e902d4b9c\": container with ID starting with e2558896947a416f13a03214314666f6b44528ed5966fc120dc63d3e902d4b9c not found: ID does not exist" Jul 06 18:53:45.413642 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.413615 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7"] Jul 06 18:53:45.420711 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.420685 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-f2348c-openshift-default-5b746f795c-mmff7"] Jul 06 18:53:45.850216 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:45.850174 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2e30913-d907-4c06-a2ab-b53e96aace80" path="/var/lib/kubelet/pods/c2e30913-d907-4c06-a2ab-b53e96aace80/volumes" Jul 06 18:53:48.012399 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:48.012366 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:53:48.137059 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:48.137023 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:53:48.317725 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:48.317629 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g"] Jul 06 18:53:48.317926 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:48.317899 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" podUID="9068b778-5a45-4466-951a-3d00db9c3833" containerName="istio-proxy" containerID="cri-o://b202355d7381f16468a8781dd3aa30b88e53ec4fc0f7e9ecabda8e7eebb39c3f" gracePeriod=30 Jul 06 18:53:48.353133 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:48.353094 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:53:48.600113 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:48.600021 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:53:48.935600 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:48.935560 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:53:49.211779 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.211695 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b"] Jul 06 18:53:49.212150 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.212126 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2e30913-d907-4c06-a2ab-b53e96aace80" containerName="istio-proxy" Jul 06 18:53:49.212150 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.212138 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2e30913-d907-4c06-a2ab-b53e96aace80" containerName="istio-proxy" Jul 06 18:53:49.212220 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.212159 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2e30913-d907-4c06-a2ab-b53e96aace80" containerName="istio-proxy" Jul 06 18:53:49.212220 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.212165 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2e30913-d907-4c06-a2ab-b53e96aace80" containerName="istio-proxy" Jul 06 18:53:49.212288 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.212224 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2e30913-d907-4c06-a2ab-b53e96aace80" containerName="istio-proxy" Jul 06 18:53:49.212288 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.212234 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2e30913-d907-4c06-a2ab-b53e96aace80" containerName="istio-proxy" Jul 06 18:53:49.216946 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.216922 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:49.219930 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.219911 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-f58a8f-openshift-default-dockercfg-glwsd\"" Jul 06 18:53:49.228803 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.228770 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b"] Jul 06 18:53:49.244271 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.244239 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:53:49.317698 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.317636 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-workload-socket\") pod \"e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:49.318024 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.317727 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istio-token\") pod \"e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:49.318024 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.317774 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istiod-ca-cert\") pod \"e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:49.318024 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.317861 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6dl2\" (UniqueName: \"kubernetes.io/projected/e8c4d260-3107-4b73-b8ee-a3ece46d6532-kube-api-access-h6dl2\") pod \"e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:49.318024 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.317923 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istio-data\") pod \"e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:49.318024 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.317958 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istio-envoy\") pod \"e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:49.318024 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.318021 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-workload-certs\") pod \"e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:49.318412 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.318052 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istio-podinfo\") pod \"e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:49.318412 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.318336 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-credential-socket\") pod \"e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:49.419322 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.419289 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istiod-ca-cert\") pod \"e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:49.419525 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.419340 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6dl2\" (UniqueName: \"kubernetes.io/projected/e8c4d260-3107-4b73-b8ee-a3ece46d6532-kube-api-access-h6dl2\") pod \"e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:49.419525 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.419374 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istio-data\") pod \"e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:49.419525 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.419401 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istio-envoy\") pod \"e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:49.419525 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.419453 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-workload-certs\") pod \"e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:49.419525 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.419480 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istio-podinfo\") pod \"e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:49.419525 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.419508 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-credential-socket\") pod \"e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:49.419924 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.419568 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-workload-socket\") pod \"e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:49.419924 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.419628 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istio-token\") pod \"e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:49.419924 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.419839 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istio-data\") pod \"e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:49.420096 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.419951 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-credential-socket\") pod \"e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:49.420096 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.419980 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-workload-certs\") pod \"e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:49.420096 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.420044 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-workload-socket\") pod \"e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:49.420217 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.420194 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istiod-ca-cert\") pod \"e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:49.422204 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.422176 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istio-envoy\") pod \"e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:49.422361 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.422343 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istio-podinfo\") pod \"e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:49.430386 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.430344 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istio-token\") pod \"e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:49.430543 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.430473 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6dl2\" (UniqueName: \"kubernetes.io/projected/e8c4d260-3107-4b73-b8ee-a3ece46d6532-kube-api-access-h6dl2\") pod \"e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:49.527645 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.527547 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:49.546564 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.546532 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:53:49.681333 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.681296 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b"] Jul 06 18:53:49.682046 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:53:49.682021 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8c4d260_3107_4b73_b8ee_a3ece46d6532.slice/crio-c0a4c21a8fa476a68e78dc23cdcffc63ae32cb8f09cde8315d0362f8e8f9e863 WatchSource:0}: Error finding container c0a4c21a8fa476a68e78dc23cdcffc63ae32cb8f09cde8315d0362f8e8f9e863: Status 404 returned error can't find the container with id c0a4c21a8fa476a68e78dc23cdcffc63ae32cb8f09cde8315d0362f8e8f9e863 Jul 06 18:53:49.684165 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.684128 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:53:49.684276 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.684216 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:53:49.684276 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:49.684260 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:53:50.213073 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:50.213037 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:53:50.339326 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:50.339298 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:53:50.420666 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:50.420630 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" event={"ID":"e8c4d260-3107-4b73-b8ee-a3ece46d6532","Type":"ContainerStarted","Data":"0f7b6d2dff9596ec0179b3a99a1d01ebe72a3994d2bb6a824ee6da05cb3aef48"} Jul 06 18:53:50.420856 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:50.420695 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" event={"ID":"e8c4d260-3107-4b73-b8ee-a3ece46d6532","Type":"ContainerStarted","Data":"c0a4c21a8fa476a68e78dc23cdcffc63ae32cb8f09cde8315d0362f8e8f9e863"} Jul 06 18:53:50.455076 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:50.455022 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" podStartSLOduration=1.455004399 podStartE2EDuration="1.455004399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:53:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:53:50.453091327 +0000 UTC m=+2047.179520326" watchObservedRunningTime="2026-07-06 18:53:50.455004399 +0000 UTC m=+2047.181433357" Jul 06 18:53:50.528153 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:50.528063 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:50.533200 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:50.533177 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:51.426540 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:51.426505 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:51.427406 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:51.427382 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:53:52.278162 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:52.278126 2566 patch_prober.go:28] interesting pod/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 18:53:52.278331 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:52.278196 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" podUID="9068b778-5a45-4466-951a-3d00db9c3833" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 18:53:52.545936 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:52.545822 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:53:52.647288 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:52.647251 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:53:52.750296 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:52.750261 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:53:53.176204 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.176165 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:53:53.330358 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.330319 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:53:53.440179 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.440147 2566 generic.go:356] "Generic (PLEG): container finished" podID="9068b778-5a45-4466-951a-3d00db9c3833" containerID="b202355d7381f16468a8781dd3aa30b88e53ec4fc0f7e9ecabda8e7eebb39c3f" exitCode=0 Jul 06 18:53:53.440362 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.440215 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" event={"ID":"9068b778-5a45-4466-951a-3d00db9c3833","Type":"ContainerDied","Data":"b202355d7381f16468a8781dd3aa30b88e53ec4fc0f7e9ecabda8e7eebb39c3f"} Jul 06 18:53:53.440362 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.440281 2566 scope.go:117] "RemoveContainer" containerID="e10bf8ba1734672c20be97cd660cb778cffe281c03b91afa1d7be4850e7dcaa1" Jul 06 18:53:53.574859 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.574826 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:53:53.663418 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.663390 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-istio-data\") pod \"9068b778-5a45-4466-951a-3d00db9c3833\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " Jul 06 18:53:53.663615 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.663428 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9068b778-5a45-4466-951a-3d00db9c3833-istio-podinfo\") pod \"9068b778-5a45-4466-951a-3d00db9c3833\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " Jul 06 18:53:53.663615 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.663458 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-workload-certs\") pod \"9068b778-5a45-4466-951a-3d00db9c3833\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " Jul 06 18:53:53.663615 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.663497 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-workload-socket\") pod \"9068b778-5a45-4466-951a-3d00db9c3833\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " Jul 06 18:53:53.663779 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.663615 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9068b778-5a45-4466-951a-3d00db9c3833-istio-token\") pod \"9068b778-5a45-4466-951a-3d00db9c3833\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " Jul 06 18:53:53.663779 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.663672 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9068b778-5a45-4466-951a-3d00db9c3833-istiod-ca-cert\") pod \"9068b778-5a45-4466-951a-3d00db9c3833\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " Jul 06 18:53:53.663779 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.663755 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9068b778-5a45-4466-951a-3d00db9c3833" (UID: "9068b778-5a45-4466-951a-3d00db9c3833"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:53:53.663779 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.663756 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-credential-socket\") pod \"9068b778-5a45-4466-951a-3d00db9c3833\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " Jul 06 18:53:53.664028 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.663779 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9068b778-5a45-4466-951a-3d00db9c3833" (UID: "9068b778-5a45-4466-951a-3d00db9c3833"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:53:53.664028 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.663807 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4kc6\" (UniqueName: \"kubernetes.io/projected/9068b778-5a45-4466-951a-3d00db9c3833-kube-api-access-s4kc6\") pod \"9068b778-5a45-4466-951a-3d00db9c3833\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " Jul 06 18:53:53.664028 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.663923 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-istio-envoy\") pod \"9068b778-5a45-4466-951a-3d00db9c3833\" (UID: \"9068b778-5a45-4466-951a-3d00db9c3833\") " Jul 06 18:53:53.664186 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.664026 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9068b778-5a45-4466-951a-3d00db9c3833-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9068b778-5a45-4466-951a-3d00db9c3833" (UID: "9068b778-5a45-4466-951a-3d00db9c3833"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:53:53.664301 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.664282 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-workload-certs\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:53:53.664359 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.664306 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-workload-socket\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:53:53.664359 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.664321 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9068b778-5a45-4466-951a-3d00db9c3833-istiod-ca-cert\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:53:53.664359 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.664282 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9068b778-5a45-4466-951a-3d00db9c3833" (UID: "9068b778-5a45-4466-951a-3d00db9c3833"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:53:53.664520 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.664409 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-istio-data" (OuterVolumeSpecName: "istio-data") pod "9068b778-5a45-4466-951a-3d00db9c3833" (UID: "9068b778-5a45-4466-951a-3d00db9c3833"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:53:53.666178 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.666156 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9068b778-5a45-4466-951a-3d00db9c3833" (UID: "9068b778-5a45-4466-951a-3d00db9c3833"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:53:53.666384 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.666368 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9068b778-5a45-4466-951a-3d00db9c3833-kube-api-access-s4kc6" (OuterVolumeSpecName: "kube-api-access-s4kc6") pod "9068b778-5a45-4466-951a-3d00db9c3833" (UID: "9068b778-5a45-4466-951a-3d00db9c3833"). InnerVolumeSpecName "kube-api-access-s4kc6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:53:53.666474 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.666458 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9068b778-5a45-4466-951a-3d00db9c3833-istio-token" (OuterVolumeSpecName: "istio-token") pod "9068b778-5a45-4466-951a-3d00db9c3833" (UID: "9068b778-5a45-4466-951a-3d00db9c3833"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:53:53.666589 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.666466 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9068b778-5a45-4466-951a-3d00db9c3833-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9068b778-5a45-4466-951a-3d00db9c3833" (UID: "9068b778-5a45-4466-951a-3d00db9c3833"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 18:53:53.765541 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.765453 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-istio-data\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:53:53.765541 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.765483 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9068b778-5a45-4466-951a-3d00db9c3833-istio-podinfo\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:53:53.765541 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.765492 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9068b778-5a45-4466-951a-3d00db9c3833-istio-token\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:53:53.765541 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.765500 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-credential-socket\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:53:53.765541 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.765508 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4kc6\" (UniqueName: \"kubernetes.io/projected/9068b778-5a45-4466-951a-3d00db9c3833-kube-api-access-s4kc6\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:53:53.765541 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:53.765516 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9068b778-5a45-4466-951a-3d00db9c3833-istio-envoy\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:53:54.447982 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:54.447952 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" Jul 06 18:53:54.448169 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:54.447956 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g" event={"ID":"9068b778-5a45-4466-951a-3d00db9c3833","Type":"ContainerDied","Data":"f83d0111e7469d3e47323b4e78d111ed637245a55390b006a367c91e2219274b"} Jul 06 18:53:54.448169 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:54.448112 2566 scope.go:117] "RemoveContainer" containerID="b202355d7381f16468a8781dd3aa30b88e53ec4fc0f7e9ecabda8e7eebb39c3f" Jul 06 18:53:54.467196 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:54.467156 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g"] Jul 06 18:53:54.471453 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:54.470904 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-ef5ef7-openshift-default-c45c544d7-5hh6g"] Jul 06 18:53:55.849151 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:55.849112 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9068b778-5a45-4466-951a-3d00db9c3833" path="/var/lib/kubelet/pods/9068b778-5a45-4466-951a-3d00db9c3833/volumes" Jul 06 18:53:57.102017 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:57.101984 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:53:57.605537 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:57.605484 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:53:57.900743 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:57.900712 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz"] Jul 06 18:53:57.901146 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:57.901132 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9068b778-5a45-4466-951a-3d00db9c3833" containerName="istio-proxy" Jul 06 18:53:57.901197 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:57.901148 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9068b778-5a45-4466-951a-3d00db9c3833" containerName="istio-proxy" Jul 06 18:53:57.901233 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:57.901200 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9068b778-5a45-4466-951a-3d00db9c3833" containerName="istio-proxy" Jul 06 18:53:57.901233 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:57.901206 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9068b778-5a45-4466-951a-3d00db9c3833" containerName="istio-proxy" Jul 06 18:53:57.901298 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:57.901272 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9068b778-5a45-4466-951a-3d00db9c3833" containerName="istio-proxy" Jul 06 18:53:57.901298 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:57.901281 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9068b778-5a45-4466-951a-3d00db9c3833" containerName="istio-proxy" Jul 06 18:53:57.905951 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:57.905929 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:57.909511 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:57.909493 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-47576a-openshift-default-dockercfg-2wh2t\"" Jul 06 18:53:57.923237 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:57.923211 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz"] Jul 06 18:53:57.934198 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:57.934169 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:53:58.006808 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.006762 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istio-token\") pod \"e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:58.007047 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.006849 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-workload-socket\") pod \"e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:58.007047 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.006951 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istio-podinfo\") pod \"e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:58.007184 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.007100 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9crwx\" (UniqueName: \"kubernetes.io/projected/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-kube-api-access-9crwx\") pod \"e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:58.007184 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.007160 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-credential-socket\") pod \"e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:58.007291 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.007199 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istiod-ca-cert\") pod \"e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:58.007291 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.007242 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istio-data\") pod \"e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:58.007415 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.007303 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-workload-certs\") pod \"e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:58.007415 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.007349 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istio-envoy\") pod \"e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:58.108298 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.108259 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-workload-socket\") pod \"e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:58.108298 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.108301 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istio-podinfo\") pod \"e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:58.108811 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.108338 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9crwx\" (UniqueName: \"kubernetes.io/projected/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-kube-api-access-9crwx\") pod \"e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:58.108811 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.108373 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-credential-socket\") pod \"e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:58.108811 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.108396 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istiod-ca-cert\") pod \"e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:58.108811 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.108425 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istio-data\") pod \"e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:58.108811 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.108576 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-workload-certs\") pod \"e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:58.108811 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.108630 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istio-envoy\") pod \"e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:58.108811 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.108662 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istio-token\") pod \"e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:58.108811 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.108725 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-workload-socket\") pod \"e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:58.109254 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.108846 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-credential-socket\") pod \"e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:58.109254 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.108953 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istio-data\") pod \"e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:58.109254 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.108977 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-workload-certs\") pod \"e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:58.109387 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.109280 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istiod-ca-cert\") pod \"e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:58.111003 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.110983 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istio-envoy\") pod \"e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:58.111064 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.111030 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istio-podinfo\") pod \"e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:58.122975 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.122947 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istio-token\") pod \"e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:58.123554 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.123528 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9crwx\" (UniqueName: \"kubernetes.io/projected/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-kube-api-access-9crwx\") pod \"e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:58.148643 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.148615 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:53:58.217143 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.217056 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:58.365757 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.365730 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz"] Jul 06 18:53:58.367842 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:53:58.367817 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod354c1db9_d4a3_42a0_84ac_fc5ef443b64e.slice/crio-7eb735e9f8d8a8e1af0a8f044cb7644137be458e5dd48641b2b33f95638527ef WatchSource:0}: Error finding container 7eb735e9f8d8a8e1af0a8f044cb7644137be458e5dd48641b2b33f95638527ef: Status 404 returned error can't find the container with id 7eb735e9f8d8a8e1af0a8f044cb7644137be458e5dd48641b2b33f95638527ef Jul 06 18:53:58.369966 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.369936 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:53:58.370049 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.370014 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:53:58.370122 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.370062 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:53:58.474716 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.474623 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" event={"ID":"354c1db9-d4a3-42a0-84ac-fc5ef443b64e","Type":"ContainerStarted","Data":"a2347b3277d62441e455e934b89e8a986240a6dae7f5eadecb81bd548a875117"} Jul 06 18:53:58.474716 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.474678 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" event={"ID":"354c1db9-d4a3-42a0-84ac-fc5ef443b64e","Type":"ContainerStarted","Data":"7eb735e9f8d8a8e1af0a8f044cb7644137be458e5dd48641b2b33f95638527ef"} Jul 06 18:53:58.496004 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:58.495955 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" podStartSLOduration=1.495940104 podStartE2EDuration="1.495940104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:53:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:53:58.495794576 +0000 UTC m=+2055.222223536" watchObservedRunningTime="2026-07-06 18:53:58.495940104 +0000 UTC m=+2055.222369061" Jul 06 18:53:59.217958 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:59.217921 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:59.223160 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:59.223133 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:59.481298 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:59.481212 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:53:59.482199 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:53:59.482181 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:54:00.579951 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:00.579918 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:00.696918 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:00.696847 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:00.779119 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:00.779080 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:00.885472 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:00.885378 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:01.524798 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:01.524682 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:01.625859 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:01.625825 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:05.067333 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:05.067243 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:07.469027 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:07.468982 2566 patch_prober.go:28] interesting pod/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 18:54:07.469572 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:07.469066 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" podUID="e8c4d260-3107-4b73-b8ee-a3ece46d6532" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 18:54:08.545443 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:08.545358 2566 generic.go:356] "Generic (PLEG): container finished" podID="e8c4d260-3107-4b73-b8ee-a3ece46d6532" containerID="0f7b6d2dff9596ec0179b3a99a1d01ebe72a3994d2bb6a824ee6da05cb3aef48" exitCode=0 Jul 06 18:54:08.545443 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:08.545412 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" event={"ID":"e8c4d260-3107-4b73-b8ee-a3ece46d6532","Type":"ContainerDied","Data":"0f7b6d2dff9596ec0179b3a99a1d01ebe72a3994d2bb6a824ee6da05cb3aef48"} Jul 06 18:54:08.545908 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:08.545809 2566 scope.go:117] "RemoveContainer" containerID="0f7b6d2dff9596ec0179b3a99a1d01ebe72a3994d2bb6a824ee6da05cb3aef48" Jul 06 18:54:08.546935 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:08.546901 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:54:08.547015 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:08.546988 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:54:08.547059 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:08.547022 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:54:09.554247 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:09.554208 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" event={"ID":"e8c4d260-3107-4b73-b8ee-a3ece46d6532","Type":"ContainerStarted","Data":"3f49d34e56576661efd20ccbad2bb68d0b51d1e1cacdc867644d08c60fcf0f52"} Jul 06 18:54:10.528671 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:10.528560 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:54:10.529133 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:10.529100 2566 patch_prober.go:28] interesting pod/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Jul 06 18:54:10.529240 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:10.529168 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" podUID="e8c4d260-3107-4b73-b8ee-a3ece46d6532" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Jul 06 18:54:11.529055 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:11.529007 2566 patch_prober.go:28] interesting pod/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Jul 06 18:54:11.529450 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:11.529096 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" podUID="e8c4d260-3107-4b73-b8ee-a3ece46d6532" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Jul 06 18:54:12.527971 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:12.527933 2566 patch_prober.go:28] interesting pod/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Jul 06 18:54:12.528156 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:12.527989 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" podUID="e8c4d260-3107-4b73-b8ee-a3ece46d6532" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Jul 06 18:54:13.283226 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:13.283181 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:13.528713 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:13.528666 2566 patch_prober.go:28] interesting pod/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Jul 06 18:54:13.528924 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:13.528741 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" podUID="e8c4d260-3107-4b73-b8ee-a3ece46d6532" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Jul 06 18:54:13.740037 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:13.739985 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:13.886982 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:13.886940 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:13.995436 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:13.995342 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:14.195625 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:14.195571 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:14.531503 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:14.531476 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:54:14.531970 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:14.531752 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:54:14.532567 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:14.532549 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:54:14.830285 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:14.830256 2566 patch_prober.go:28] interesting pod/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": read tcp 10.134.0.2:44162->10.134.0.65:15021: read: connection reset by peer" start-of-body= Jul 06 18:54:14.830426 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:14.830336 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" podUID="354c1db9-d4a3-42a0-84ac-fc5ef443b64e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": read tcp 10.134.0.2:44162->10.134.0.65:15021: read: connection reset by peer" Jul 06 18:54:15.593994 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:15.593959 2566 generic.go:356] "Generic (PLEG): container finished" podID="354c1db9-d4a3-42a0-84ac-fc5ef443b64e" containerID="a2347b3277d62441e455e934b89e8a986240a6dae7f5eadecb81bd548a875117" exitCode=0 Jul 06 18:54:15.594383 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:15.594040 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" event={"ID":"354c1db9-d4a3-42a0-84ac-fc5ef443b64e","Type":"ContainerDied","Data":"a2347b3277d62441e455e934b89e8a986240a6dae7f5eadecb81bd548a875117"} Jul 06 18:54:15.594545 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:15.594528 2566 scope.go:117] "RemoveContainer" containerID="a2347b3277d62441e455e934b89e8a986240a6dae7f5eadecb81bd548a875117" Jul 06 18:54:15.595562 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:15.595535 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:54:15.595618 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:15.595601 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:54:15.595660 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:15.595631 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:54:16.640196 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:16.640157 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" event={"ID":"354c1db9-d4a3-42a0-84ac-fc5ef443b64e","Type":"ContainerStarted","Data":"ca5053c598bd9e9986616a933adaf16238e292e98f5c84aa02ca372a4d2ca927"} Jul 06 18:54:17.218361 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:17.218325 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:54:17.218778 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:17.218668 2566 patch_prober.go:28] interesting pod/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body= Jul 06 18:54:17.219005 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:17.218969 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" podUID="354c1db9-d4a3-42a0-84ac-fc5ef443b64e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" Jul 06 18:54:18.217685 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:18.217632 2566 patch_prober.go:28] interesting pod/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body= Jul 06 18:54:18.218246 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:18.217732 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" podUID="354c1db9-d4a3-42a0-84ac-fc5ef443b64e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" Jul 06 18:54:19.218252 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:19.218211 2566 patch_prober.go:28] interesting pod/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body= Jul 06 18:54:19.218761 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:19.218306 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" podUID="354c1db9-d4a3-42a0-84ac-fc5ef443b64e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" Jul 06 18:54:19.738106 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:19.738054 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:19.810039 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:19.810006 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:19.978988 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:19.978953 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b"] Jul 06 18:54:19.979260 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:19.979233 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" podUID="e8c4d260-3107-4b73-b8ee-a3ece46d6532" containerName="istio-proxy" containerID="cri-o://3f49d34e56576661efd20ccbad2bb68d0b51d1e1cacdc867644d08c60fcf0f52" gracePeriod=30 Jul 06 18:54:20.005955 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:20.002680 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:20.218715 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:20.218663 2566 patch_prober.go:28] interesting pod/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body= Jul 06 18:54:20.219219 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:20.218986 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" podUID="354c1db9-d4a3-42a0-84ac-fc5ef443b64e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" Jul 06 18:54:20.260009 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:20.259914 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:20.381593 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:20.381561 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:20.774217 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:20.774181 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:20.880854 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:20.880823 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:21.032811 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:21.032716 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:21.221119 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:21.221082 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:54:21.221516 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:21.221360 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:54:21.221992 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:21.221976 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:54:25.233843 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.233817 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:54:25.409462 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.409434 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istio-envoy\") pod \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " Jul 06 18:54:25.409665 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.409469 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-workload-socket\") pod \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " Jul 06 18:54:25.409665 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.409513 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istio-podinfo\") pod \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " Jul 06 18:54:25.409665 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.409550 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-credential-socket\") pod \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " Jul 06 18:54:25.409665 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.409569 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6dl2\" (UniqueName: \"kubernetes.io/projected/e8c4d260-3107-4b73-b8ee-a3ece46d6532-kube-api-access-h6dl2\") pod \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " Jul 06 18:54:25.409665 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.409601 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istio-data\") pod \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " Jul 06 18:54:25.409665 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.409654 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istio-token\") pod \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " Jul 06 18:54:25.410030 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.409690 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istiod-ca-cert\") pod \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " Jul 06 18:54:25.410030 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.409743 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-workload-certs\") pod \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\" (UID: \"e8c4d260-3107-4b73-b8ee-a3ece46d6532\") " Jul 06 18:54:25.410030 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.409810 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e8c4d260-3107-4b73-b8ee-a3ece46d6532" (UID: "e8c4d260-3107-4b73-b8ee-a3ece46d6532"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:54:25.410187 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.410125 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e8c4d260-3107-4b73-b8ee-a3ece46d6532" (UID: "e8c4d260-3107-4b73-b8ee-a3ece46d6532"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:54:25.410290 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.410194 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-workload-socket\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:54:25.410535 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.410298 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e8c4d260-3107-4b73-b8ee-a3ece46d6532" (UID: "e8c4d260-3107-4b73-b8ee-a3ece46d6532"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:54:25.410616 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.410515 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e8c4d260-3107-4b73-b8ee-a3ece46d6532" (UID: "e8c4d260-3107-4b73-b8ee-a3ece46d6532"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:54:25.410908 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.410857 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istio-data" (OuterVolumeSpecName: "istio-data") pod "e8c4d260-3107-4b73-b8ee-a3ece46d6532" (UID: "e8c4d260-3107-4b73-b8ee-a3ece46d6532"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:54:25.412037 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.411999 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8c4d260-3107-4b73-b8ee-a3ece46d6532-kube-api-access-h6dl2" (OuterVolumeSpecName: "kube-api-access-h6dl2") pod "e8c4d260-3107-4b73-b8ee-a3ece46d6532" (UID: "e8c4d260-3107-4b73-b8ee-a3ece46d6532"). InnerVolumeSpecName "kube-api-access-h6dl2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:54:25.412037 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.412016 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e8c4d260-3107-4b73-b8ee-a3ece46d6532" (UID: "e8c4d260-3107-4b73-b8ee-a3ece46d6532"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:54:25.412172 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.412114 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istio-token" (OuterVolumeSpecName: "istio-token") pod "e8c4d260-3107-4b73-b8ee-a3ece46d6532" (UID: "e8c4d260-3107-4b73-b8ee-a3ece46d6532"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:54:25.412172 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.412117 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e8c4d260-3107-4b73-b8ee-a3ece46d6532" (UID: "e8c4d260-3107-4b73-b8ee-a3ece46d6532"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 18:54:25.511685 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.511645 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istio-envoy\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:54:25.511685 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.511681 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istio-podinfo\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:54:25.511685 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.511693 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-credential-socket\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:54:25.511961 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.511706 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6dl2\" (UniqueName: \"kubernetes.io/projected/e8c4d260-3107-4b73-b8ee-a3ece46d6532-kube-api-access-h6dl2\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:54:25.511961 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.511717 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istio-data\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:54:25.511961 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.511728 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istio-token\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:54:25.511961 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.511739 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8c4d260-3107-4b73-b8ee-a3ece46d6532-istiod-ca-cert\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:54:25.511961 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.511749 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8c4d260-3107-4b73-b8ee-a3ece46d6532-workload-certs\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:54:25.694000 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.693911 2566 generic.go:356] "Generic (PLEG): container finished" podID="e8c4d260-3107-4b73-b8ee-a3ece46d6532" containerID="3f49d34e56576661efd20ccbad2bb68d0b51d1e1cacdc867644d08c60fcf0f52" exitCode=0 Jul 06 18:54:25.694000 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.693961 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" Jul 06 18:54:25.694182 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.694007 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" event={"ID":"e8c4d260-3107-4b73-b8ee-a3ece46d6532","Type":"ContainerDied","Data":"3f49d34e56576661efd20ccbad2bb68d0b51d1e1cacdc867644d08c60fcf0f52"} Jul 06 18:54:25.694182 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.694062 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b" event={"ID":"e8c4d260-3107-4b73-b8ee-a3ece46d6532","Type":"ContainerDied","Data":"c0a4c21a8fa476a68e78dc23cdcffc63ae32cb8f09cde8315d0362f8e8f9e863"} Jul 06 18:54:25.694182 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.694092 2566 scope.go:117] "RemoveContainer" containerID="3f49d34e56576661efd20ccbad2bb68d0b51d1e1cacdc867644d08c60fcf0f52" Jul 06 18:54:25.704374 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.704358 2566 scope.go:117] "RemoveContainer" containerID="0f7b6d2dff9596ec0179b3a99a1d01ebe72a3994d2bb6a824ee6da05cb3aef48" Jul 06 18:54:25.713496 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.713475 2566 scope.go:117] "RemoveContainer" containerID="3f49d34e56576661efd20ccbad2bb68d0b51d1e1cacdc867644d08c60fcf0f52" Jul 06 18:54:25.713781 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:54:25.713761 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f49d34e56576661efd20ccbad2bb68d0b51d1e1cacdc867644d08c60fcf0f52\": container with ID starting with 3f49d34e56576661efd20ccbad2bb68d0b51d1e1cacdc867644d08c60fcf0f52 not found: ID does not exist" containerID="3f49d34e56576661efd20ccbad2bb68d0b51d1e1cacdc867644d08c60fcf0f52" Jul 06 18:54:25.713837 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.713791 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f49d34e56576661efd20ccbad2bb68d0b51d1e1cacdc867644d08c60fcf0f52"} err="failed to get container status \"3f49d34e56576661efd20ccbad2bb68d0b51d1e1cacdc867644d08c60fcf0f52\": rpc error: code = NotFound desc = could not find container \"3f49d34e56576661efd20ccbad2bb68d0b51d1e1cacdc867644d08c60fcf0f52\": container with ID starting with 3f49d34e56576661efd20ccbad2bb68d0b51d1e1cacdc867644d08c60fcf0f52 not found: ID does not exist" Jul 06 18:54:25.713837 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.713809 2566 scope.go:117] "RemoveContainer" containerID="0f7b6d2dff9596ec0179b3a99a1d01ebe72a3994d2bb6a824ee6da05cb3aef48" Jul 06 18:54:25.714052 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:54:25.714034 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f7b6d2dff9596ec0179b3a99a1d01ebe72a3994d2bb6a824ee6da05cb3aef48\": container with ID starting with 0f7b6d2dff9596ec0179b3a99a1d01ebe72a3994d2bb6a824ee6da05cb3aef48 not found: ID does not exist" containerID="0f7b6d2dff9596ec0179b3a99a1d01ebe72a3994d2bb6a824ee6da05cb3aef48" Jul 06 18:54:25.714098 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.714060 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f7b6d2dff9596ec0179b3a99a1d01ebe72a3994d2bb6a824ee6da05cb3aef48"} err="failed to get container status \"0f7b6d2dff9596ec0179b3a99a1d01ebe72a3994d2bb6a824ee6da05cb3aef48\": rpc error: code = NotFound desc = could not find container \"0f7b6d2dff9596ec0179b3a99a1d01ebe72a3994d2bb6a824ee6da05cb3aef48\": container with ID starting with 0f7b6d2dff9596ec0179b3a99a1d01ebe72a3994d2bb6a824ee6da05cb3aef48 not found: ID does not exist" Jul 06 18:54:25.719778 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.719758 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b"] Jul 06 18:54:25.724135 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.724111 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-f58a8f-openshift-default-c4d4f4d9f-8mb8b"] Jul 06 18:54:25.849793 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:25.849756 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8c4d260-3107-4b73-b8ee-a3ece46d6532" path="/var/lib/kubelet/pods/e8c4d260-3107-4b73-b8ee-a3ece46d6532/volumes" Jul 06 18:54:26.887723 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:26.887689 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:27.010150 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:27.010120 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:27.113292 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:27.113262 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz"] Jul 06 18:54:27.113485 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:27.113457 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" podUID="354c1db9-d4a3-42a0-84ac-fc5ef443b64e" containerName="istio-proxy" containerID="cri-o://ca5053c598bd9e9986616a933adaf16238e292e98f5c84aa02ca372a4d2ca927" gracePeriod=30 Jul 06 18:54:27.128296 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:27.128267 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:27.444511 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:27.444481 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:27.832588 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:27.832502 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:28.136960 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.136877 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm"] Jul 06 18:54:28.137323 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.137282 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8c4d260-3107-4b73-b8ee-a3ece46d6532" containerName="istio-proxy" Jul 06 18:54:28.137323 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.137293 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8c4d260-3107-4b73-b8ee-a3ece46d6532" containerName="istio-proxy" Jul 06 18:54:28.137323 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.137301 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8c4d260-3107-4b73-b8ee-a3ece46d6532" containerName="istio-proxy" Jul 06 18:54:28.137323 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.137307 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8c4d260-3107-4b73-b8ee-a3ece46d6532" containerName="istio-proxy" Jul 06 18:54:28.137464 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.137377 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8c4d260-3107-4b73-b8ee-a3ece46d6532" containerName="istio-proxy" Jul 06 18:54:28.137464 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.137386 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8c4d260-3107-4b73-b8ee-a3ece46d6532" containerName="istio-proxy" Jul 06 18:54:28.140692 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.140674 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:28.155155 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.155133 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-47f1eb-openshift-default-dockercfg-2t24c\"" Jul 06 18:54:28.173250 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.173214 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm"] Jul 06 18:54:28.238351 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.238318 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istio-data\") pod \"e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:28.238568 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.238365 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-credential-socket\") pod \"e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:28.238568 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.238388 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlx67\" (UniqueName: \"kubernetes.io/projected/0b104471-f353-43b3-9beb-ee6c0d32dcd2-kube-api-access-vlx67\") pod \"e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:28.238568 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.238474 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-workload-certs\") pod \"e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:28.238568 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.238535 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istiod-ca-cert\") pod \"e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:28.238771 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.238593 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istio-token\") pod \"e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:28.238771 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.238648 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istio-podinfo\") pod \"e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:28.238771 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.238671 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istio-envoy\") pod \"e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:28.238771 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.238704 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-workload-socket\") pod \"e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:28.250012 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.249983 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:28.339425 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.339378 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-credential-socket\") pod \"e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:28.339425 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.339425 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlx67\" (UniqueName: \"kubernetes.io/projected/0b104471-f353-43b3-9beb-ee6c0d32dcd2-kube-api-access-vlx67\") pod \"e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:28.339693 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.339469 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-workload-certs\") pod \"e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:28.339693 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.339516 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istiod-ca-cert\") pod \"e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:28.339693 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.339554 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istio-token\") pod \"e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:28.339693 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.339601 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istio-podinfo\") pod \"e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:28.339693 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.339628 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istio-envoy\") pod \"e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:28.339693 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.339667 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-workload-socket\") pod \"e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:28.339997 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.339734 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istio-data\") pod \"e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:28.339997 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.339859 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-credential-socket\") pod \"e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:28.339997 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.339940 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-workload-certs\") pod \"e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:28.340147 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.340106 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istio-data\") pod \"e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:28.340244 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.340223 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-workload-socket\") pod \"e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:28.340354 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.340336 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istiod-ca-cert\") pod \"e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:28.342279 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.342254 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istio-envoy\") pod \"e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:28.342514 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.342491 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istio-podinfo\") pod \"e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:28.359782 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.359751 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istio-token\") pod \"e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:28.361192 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.361168 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlx67\" (UniqueName: \"kubernetes.io/projected/0b104471-f353-43b3-9beb-ee6c0d32dcd2-kube-api-access-vlx67\") pod \"e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:28.387786 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.387700 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:28.451618 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.451577 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:28.601770 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.601740 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm"] Jul 06 18:54:28.603295 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:54:28.603273 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b104471_f353_43b3_9beb_ee6c0d32dcd2.slice/crio-5f5283bc96eea0a0eaaeb8121eab4b94eb7ed16e4d5d31fdfa20f41fb693615c WatchSource:0}: Error finding container 5f5283bc96eea0a0eaaeb8121eab4b94eb7ed16e4d5d31fdfa20f41fb693615c: Status 404 returned error can't find the container with id 5f5283bc96eea0a0eaaeb8121eab4b94eb7ed16e4d5d31fdfa20f41fb693615c Jul 06 18:54:28.605524 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.605492 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:54:28.605583 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.605557 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:54:28.605623 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.605585 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:54:28.717098 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.717059 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" event={"ID":"0b104471-f353-43b3-9beb-ee6c0d32dcd2","Type":"ContainerStarted","Data":"4b86e0428a469a1b0d044491c552486eeba82f02ad2343a526ee63ffd4f8714d"} Jul 06 18:54:28.717294 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.717109 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" event={"ID":"0b104471-f353-43b3-9beb-ee6c0d32dcd2","Type":"ContainerStarted","Data":"5f5283bc96eea0a0eaaeb8121eab4b94eb7ed16e4d5d31fdfa20f41fb693615c"} Jul 06 18:54:28.745879 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:28.745787 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" podStartSLOduration=0.745766193 podStartE2EDuration="745.766193ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:54:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:54:28.741668488 +0000 UTC m=+2085.468097447" watchObservedRunningTime="2026-07-06 18:54:28.745766193 +0000 UTC m=+2085.472195151" Jul 06 18:54:29.452456 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:29.452419 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:29.457735 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:29.457710 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:29.723341 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:29.723253 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:29.724313 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:29.724287 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:30.774176 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:30.774140 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:30.853494 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:30.853457 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:30.929989 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:30.929948 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:31.299095 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:31.299049 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:31.459380 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:31.459345 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:31.595451 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:31.595341 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:32.372148 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.372125 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:54:32.481771 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.481735 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-workload-socket\") pod \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " Jul 06 18:54:32.481992 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.481814 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-workload-certs\") pod \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " Jul 06 18:54:32.481992 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.481842 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istio-data\") pod \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " Jul 06 18:54:32.481992 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.481897 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istio-podinfo\") pod \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " Jul 06 18:54:32.481992 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.481952 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9crwx\" (UniqueName: \"kubernetes.io/projected/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-kube-api-access-9crwx\") pod \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " Jul 06 18:54:32.482225 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.482001 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istio-token\") pod \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " Jul 06 18:54:32.482225 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.482046 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istiod-ca-cert\") pod \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " Jul 06 18:54:32.482225 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.482071 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-credential-socket\") pod \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " Jul 06 18:54:32.482225 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.482074 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "354c1db9-d4a3-42a0-84ac-fc5ef443b64e" (UID: "354c1db9-d4a3-42a0-84ac-fc5ef443b64e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:54:32.482225 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.482099 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istio-envoy\") pod \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\" (UID: \"354c1db9-d4a3-42a0-84ac-fc5ef443b64e\") " Jul 06 18:54:32.482225 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.482156 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "354c1db9-d4a3-42a0-84ac-fc5ef443b64e" (UID: "354c1db9-d4a3-42a0-84ac-fc5ef443b64e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:54:32.482530 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.482380 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "354c1db9-d4a3-42a0-84ac-fc5ef443b64e" (UID: "354c1db9-d4a3-42a0-84ac-fc5ef443b64e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:54:32.482530 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.482431 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-workload-socket\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:54:32.482530 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.482433 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "354c1db9-d4a3-42a0-84ac-fc5ef443b64e" (UID: "354c1db9-d4a3-42a0-84ac-fc5ef443b64e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:54:32.482530 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.482447 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-workload-certs\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:54:32.482914 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.482859 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istio-data" (OuterVolumeSpecName: "istio-data") pod "354c1db9-d4a3-42a0-84ac-fc5ef443b64e" (UID: "354c1db9-d4a3-42a0-84ac-fc5ef443b64e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:54:32.484433 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.484413 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "354c1db9-d4a3-42a0-84ac-fc5ef443b64e" (UID: "354c1db9-d4a3-42a0-84ac-fc5ef443b64e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 18:54:32.484510 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.484464 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "354c1db9-d4a3-42a0-84ac-fc5ef443b64e" (UID: "354c1db9-d4a3-42a0-84ac-fc5ef443b64e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:54:32.484675 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.484655 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-kube-api-access-9crwx" (OuterVolumeSpecName: "kube-api-access-9crwx") pod "354c1db9-d4a3-42a0-84ac-fc5ef443b64e" (UID: "354c1db9-d4a3-42a0-84ac-fc5ef443b64e"). InnerVolumeSpecName "kube-api-access-9crwx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:54:32.484774 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.484756 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istio-token" (OuterVolumeSpecName: "istio-token") pod "354c1db9-d4a3-42a0-84ac-fc5ef443b64e" (UID: "354c1db9-d4a3-42a0-84ac-fc5ef443b64e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:54:32.583584 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.583542 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istio-token\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:54:32.583584 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.583576 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istiod-ca-cert\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:54:32.583808 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.583600 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-credential-socket\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:54:32.583808 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.583609 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istio-envoy\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:54:32.583808 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.583618 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istio-data\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:54:32.583808 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.583626 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-istio-podinfo\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:54:32.583808 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.583634 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9crwx\" (UniqueName: \"kubernetes.io/projected/354c1db9-d4a3-42a0-84ac-fc5ef443b64e-kube-api-access-9crwx\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 18:54:32.747075 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.746983 2566 generic.go:356] "Generic (PLEG): container finished" podID="354c1db9-d4a3-42a0-84ac-fc5ef443b64e" containerID="ca5053c598bd9e9986616a933adaf16238e292e98f5c84aa02ca372a4d2ca927" exitCode=0 Jul 06 18:54:32.747075 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.747031 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" Jul 06 18:54:32.747319 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.747078 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" event={"ID":"354c1db9-d4a3-42a0-84ac-fc5ef443b64e","Type":"ContainerDied","Data":"ca5053c598bd9e9986616a933adaf16238e292e98f5c84aa02ca372a4d2ca927"} Jul 06 18:54:32.747319 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.747139 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz" event={"ID":"354c1db9-d4a3-42a0-84ac-fc5ef443b64e","Type":"ContainerDied","Data":"7eb735e9f8d8a8e1af0a8f044cb7644137be458e5dd48641b2b33f95638527ef"} Jul 06 18:54:32.747319 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.747167 2566 scope.go:117] "RemoveContainer" containerID="ca5053c598bd9e9986616a933adaf16238e292e98f5c84aa02ca372a4d2ca927" Jul 06 18:54:32.757333 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.757317 2566 scope.go:117] "RemoveContainer" containerID="a2347b3277d62441e455e934b89e8a986240a6dae7f5eadecb81bd548a875117" Jul 06 18:54:32.766634 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.766615 2566 scope.go:117] "RemoveContainer" containerID="ca5053c598bd9e9986616a933adaf16238e292e98f5c84aa02ca372a4d2ca927" Jul 06 18:54:32.766916 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:54:32.766898 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca5053c598bd9e9986616a933adaf16238e292e98f5c84aa02ca372a4d2ca927\": container with ID starting with ca5053c598bd9e9986616a933adaf16238e292e98f5c84aa02ca372a4d2ca927 not found: ID does not exist" containerID="ca5053c598bd9e9986616a933adaf16238e292e98f5c84aa02ca372a4d2ca927" Jul 06 18:54:32.766986 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.766924 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca5053c598bd9e9986616a933adaf16238e292e98f5c84aa02ca372a4d2ca927"} err="failed to get container status \"ca5053c598bd9e9986616a933adaf16238e292e98f5c84aa02ca372a4d2ca927\": rpc error: code = NotFound desc = could not find container \"ca5053c598bd9e9986616a933adaf16238e292e98f5c84aa02ca372a4d2ca927\": container with ID starting with ca5053c598bd9e9986616a933adaf16238e292e98f5c84aa02ca372a4d2ca927 not found: ID does not exist" Jul 06 18:54:32.766986 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.766941 2566 scope.go:117] "RemoveContainer" containerID="a2347b3277d62441e455e934b89e8a986240a6dae7f5eadecb81bd548a875117" Jul 06 18:54:32.767181 ip-10-0-128-98 kubenswrapper[2566]: E0706 18:54:32.767161 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2347b3277d62441e455e934b89e8a986240a6dae7f5eadecb81bd548a875117\": container with ID starting with a2347b3277d62441e455e934b89e8a986240a6dae7f5eadecb81bd548a875117 not found: ID does not exist" containerID="a2347b3277d62441e455e934b89e8a986240a6dae7f5eadecb81bd548a875117" Jul 06 18:54:32.767231 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.767186 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2347b3277d62441e455e934b89e8a986240a6dae7f5eadecb81bd548a875117"} err="failed to get container status \"a2347b3277d62441e455e934b89e8a986240a6dae7f5eadecb81bd548a875117\": rpc error: code = NotFound desc = could not find container \"a2347b3277d62441e455e934b89e8a986240a6dae7f5eadecb81bd548a875117\": container with ID starting with a2347b3277d62441e455e934b89e8a986240a6dae7f5eadecb81bd548a875117 not found: ID does not exist" Jul 06 18:54:32.774353 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.774332 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz"] Jul 06 18:54:32.783949 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:32.783928 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-47576a-openshift-default-77cc8f94df-vjbfz"] Jul 06 18:54:33.849724 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:33.849686 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="354c1db9-d4a3-42a0-84ac-fc5ef443b64e" path="/var/lib/kubelet/pods/354c1db9-d4a3-42a0-84ac-fc5ef443b64e/volumes" Jul 06 18:54:35.290958 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:35.290916 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:35.619259 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:35.619169 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:35.914301 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:35.914274 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6"] Jul 06 18:54:35.914660 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:35.914648 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="354c1db9-d4a3-42a0-84ac-fc5ef443b64e" containerName="istio-proxy" Jul 06 18:54:35.914736 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:35.914662 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="354c1db9-d4a3-42a0-84ac-fc5ef443b64e" containerName="istio-proxy" Jul 06 18:54:35.914736 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:35.914708 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="354c1db9-d4a3-42a0-84ac-fc5ef443b64e" containerName="istio-proxy" Jul 06 18:54:35.914736 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:35.914718 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="354c1db9-d4a3-42a0-84ac-fc5ef443b64e" containerName="istio-proxy" Jul 06 18:54:35.914920 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:35.914824 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="354c1db9-d4a3-42a0-84ac-fc5ef443b64e" containerName="istio-proxy" Jul 06 18:54:35.919320 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:35.919299 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:35.922239 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:35.922220 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-b19850-openshift-default-dockercfg-h7xhp\"" Jul 06 18:54:35.930475 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:35.930448 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6"] Jul 06 18:54:35.948246 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:35.948210 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:36.014623 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.014568 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-credential-socket\") pod \"e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:36.014623 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.014622 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92a2600a-54f0-4228-8288-aee9e1b9da45-istio-token\") pod \"e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:36.014935 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.014656 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-istio-data\") pod \"e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:36.014935 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.014715 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-istio-envoy\") pod \"e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:36.014935 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.014754 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92a2600a-54f0-4228-8288-aee9e1b9da45-istiod-ca-cert\") pod \"e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:36.014935 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.014779 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-workload-certs\") pod \"e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:36.014935 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.014853 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-workload-socket\") pod \"e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:36.014935 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.014931 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92a2600a-54f0-4228-8288-aee9e1b9da45-istio-podinfo\") pod \"e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:36.015244 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.014959 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd5xk\" (UniqueName: \"kubernetes.io/projected/92a2600a-54f0-4228-8288-aee9e1b9da45-kube-api-access-xd5xk\") pod \"e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:36.116369 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.116326 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-workload-socket\") pod \"e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:36.116572 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.116380 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92a2600a-54f0-4228-8288-aee9e1b9da45-istio-podinfo\") pod \"e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:36.116572 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.116417 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd5xk\" (UniqueName: \"kubernetes.io/projected/92a2600a-54f0-4228-8288-aee9e1b9da45-kube-api-access-xd5xk\") pod \"e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:36.116572 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.116472 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-credential-socket\") pod \"e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:36.116572 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.116498 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92a2600a-54f0-4228-8288-aee9e1b9da45-istio-token\") pod \"e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:36.116572 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.116525 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-istio-data\") pod \"e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:36.116572 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.116563 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-istio-envoy\") pod \"e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:36.116946 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.116600 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92a2600a-54f0-4228-8288-aee9e1b9da45-istiod-ca-cert\") pod \"e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:36.116946 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.116634 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-workload-certs\") pod \"e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:36.116946 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.116863 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-credential-socket\") pod \"e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:36.117058 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.116995 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-istio-data\") pod \"e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:36.117101 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.117072 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-workload-socket\") pod \"e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:36.117137 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.117115 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-workload-certs\") pod \"e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:36.117678 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.117643 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92a2600a-54f0-4228-8288-aee9e1b9da45-istiod-ca-cert\") pod \"e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:36.118970 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.118941 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-istio-envoy\") pod \"e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:36.119207 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.119190 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92a2600a-54f0-4228-8288-aee9e1b9da45-istio-podinfo\") pod \"e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:36.126764 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.126740 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92a2600a-54f0-4228-8288-aee9e1b9da45-istio-token\") pod \"e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:36.127118 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.127096 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd5xk\" (UniqueName: \"kubernetes.io/projected/92a2600a-54f0-4228-8288-aee9e1b9da45-kube-api-access-xd5xk\") pod \"e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:36.197218 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.197106 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:36.231275 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.231240 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:36.384432 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.384386 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6"] Jul 06 18:54:36.384845 ip-10-0-128-98 kubenswrapper[2566]: W0706 18:54:36.384827 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92a2600a_54f0_4228_8288_aee9e1b9da45.slice/crio-9c75185c63e153dc10ff6ffd3bf280bda20ac610c2fa9077ae5a0903cea5caea WatchSource:0}: Error finding container 9c75185c63e153dc10ff6ffd3bf280bda20ac610c2fa9077ae5a0903cea5caea: Status 404 returned error can't find the container with id 9c75185c63e153dc10ff6ffd3bf280bda20ac610c2fa9077ae5a0903cea5caea Jul 06 18:54:36.387084 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.387051 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:54:36.387184 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.387117 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:54:36.387184 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.387163 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:54:36.772112 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.772003 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" event={"ID":"92a2600a-54f0-4228-8288-aee9e1b9da45","Type":"ContainerStarted","Data":"bb80e3f1913edee254ecce6deda38fe25d0d85c49d79fda1c329a2128ba0eb3b"} Jul 06 18:54:36.772112 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.772081 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" event={"ID":"92a2600a-54f0-4228-8288-aee9e1b9da45","Type":"ContainerStarted","Data":"9c75185c63e153dc10ff6ffd3bf280bda20ac610c2fa9077ae5a0903cea5caea"} Jul 06 18:54:36.797210 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:36.797154 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" podStartSLOduration=1.7971327270000002 podStartE2EDuration="1.797132727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:54:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:54:36.795024077 +0000 UTC m=+2093.521453039" watchObservedRunningTime="2026-07-06 18:54:36.797132727 +0000 UTC m=+2093.523561685" Jul 06 18:54:37.232419 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:37.232384 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:37.237642 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:37.237611 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:37.778728 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:37.778686 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:37.779638 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:37.779621 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:38.886197 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:38.886157 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:38.989084 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:38.989046 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:39.209483 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:39.209412 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:39.335436 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:39.335402 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:39.780815 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:39.780777 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:39.882719 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:39.882677 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:43.361776 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:43.361730 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:43.969024 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:43.968985 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffzjr_1ede441d-79f2-4365-8a08-403bf043ef73/ovn-acl-logging/0.log" Jul 06 18:54:43.971178 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:43.971150 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffzjr_1ede441d-79f2-4365-8a08-403bf043ef73/ovn-acl-logging/0.log" Jul 06 18:54:44.724493 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:44.724455 2566 patch_prober.go:28] interesting pod/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Jul 06 18:54:44.724948 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:44.724522 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" podUID="0b104471-f353-43b3-9beb-ee6c0d32dcd2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Jul 06 18:54:44.828201 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:44.828169 2566 generic.go:356] "Generic (PLEG): container finished" podID="0b104471-f353-43b3-9beb-ee6c0d32dcd2" containerID="4b86e0428a469a1b0d044491c552486eeba82f02ad2343a526ee63ffd4f8714d" exitCode=0 Jul 06 18:54:44.828404 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:44.828250 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" event={"ID":"0b104471-f353-43b3-9beb-ee6c0d32dcd2","Type":"ContainerDied","Data":"4b86e0428a469a1b0d044491c552486eeba82f02ad2343a526ee63ffd4f8714d"} Jul 06 18:54:44.828741 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:44.828723 2566 scope.go:117] "RemoveContainer" containerID="4b86e0428a469a1b0d044491c552486eeba82f02ad2343a526ee63ffd4f8714d" Jul 06 18:54:44.829760 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:44.829731 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:54:44.829826 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:44.829804 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:54:44.829878 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:44.829835 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:54:45.842063 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:45.842006 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" event={"ID":"0b104471-f353-43b3-9beb-ee6c0d32dcd2","Type":"ContainerStarted","Data":"dbf6a3853e49072c183131f7888f597096932f738e75644b8eef5effb01a8ebd"} Jul 06 18:54:46.452620 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:46.452577 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:46.453001 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:46.452975 2566 patch_prober.go:28] interesting pod/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Jul 06 18:54:46.453130 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:46.453035 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" podUID="0b104471-f353-43b3-9beb-ee6c0d32dcd2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Jul 06 18:54:47.452495 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:47.452455 2566 patch_prober.go:28] interesting pod/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Jul 06 18:54:47.452930 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:47.452512 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" podUID="0b104471-f353-43b3-9beb-ee6c0d32dcd2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Jul 06 18:54:48.452581 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:48.452541 2566 patch_prober.go:28] interesting pod/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Jul 06 18:54:48.453076 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:48.452600 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" podUID="0b104471-f353-43b3-9beb-ee6c0d32dcd2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Jul 06 18:54:49.452602 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:49.452549 2566 patch_prober.go:28] interesting pod/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Jul 06 18:54:49.453170 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:49.452609 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" podUID="0b104471-f353-43b3-9beb-ee6c0d32dcd2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Jul 06 18:54:50.457400 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:50.457372 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:50.457821 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:50.457797 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:50.458764 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:50.458742 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 18:54:50.874202 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:50.874106 2566 generic.go:356] "Generic (PLEG): container finished" podID="92a2600a-54f0-4228-8288-aee9e1b9da45" containerID="bb80e3f1913edee254ecce6deda38fe25d0d85c49d79fda1c329a2128ba0eb3b" exitCode=0 Jul 06 18:54:50.874202 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:50.874181 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" event={"ID":"92a2600a-54f0-4228-8288-aee9e1b9da45","Type":"ContainerDied","Data":"bb80e3f1913edee254ecce6deda38fe25d0d85c49d79fda1c329a2128ba0eb3b"} Jul 06 18:54:50.874747 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:50.874734 2566 scope.go:117] "RemoveContainer" containerID="bb80e3f1913edee254ecce6deda38fe25d0d85c49d79fda1c329a2128ba0eb3b" Jul 06 18:54:50.875722 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:50.875689 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:54:50.875825 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:50.875782 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:54:50.875911 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:50.875831 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 18:54:51.881813 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:51.881778 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" event={"ID":"92a2600a-54f0-4228-8288-aee9e1b9da45","Type":"ContainerStarted","Data":"5542d250dbfc9858de0076edce9686c5b8f1c96ef22b9e3b4b638930e6b17f69"} Jul 06 18:54:52.231884 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:52.231832 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:52.232941 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:52.232908 2566 patch_prober.go:28] interesting pod/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Jul 06 18:54:52.233101 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:52.232971 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" podUID="92a2600a-54f0-4228-8288-aee9e1b9da45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Jul 06 18:54:52.290125 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:52.290094 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:52.390733 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:52.390698 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:54:52.779386 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:52.779346 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:53.231909 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:53.231844 2566 patch_prober.go:28] interesting pod/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Jul 06 18:54:53.232312 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:53.231935 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" podUID="92a2600a-54f0-4228-8288-aee9e1b9da45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Jul 06 18:54:54.232515 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:54.232477 2566 patch_prober.go:28] interesting pod/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Jul 06 18:54:54.232958 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:54.232535 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" podUID="92a2600a-54f0-4228-8288-aee9e1b9da45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Jul 06 18:54:55.232369 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:55.232336 2566 patch_prober.go:28] interesting pod/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Jul 06 18:54:55.232546 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:55.232388 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" podUID="92a2600a-54f0-4228-8288-aee9e1b9da45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Jul 06 18:54:56.236123 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:56.236089 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:56.237187 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:56.237168 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 18:54:58.698825 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:54:58.698784 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:00.554732 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:55:00.554699 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:00.658288 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:55:00.658253 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:28.249429 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:55:28.249389 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:55:34.995264 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:55:34.995179 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:56:48.322931 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:56:48.322886 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:57:08.313442 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:57:08.313362 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:57:08.414361 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:57:08.414325 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:57:36.177403 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:57:36.177349 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:57:37.226215 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:57:37.226167 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:57:42.157226 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:57:42.157191 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:58:12.853818 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:58:12.853776 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:58:14.836308 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:58:14.836272 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:58:20.229797 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:58:20.229754 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:58:51.237842 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:58:51.237799 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:58:51.389664 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:58:51.389621 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:58:56.067983 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:58:56.067953 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:58:57.077001 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:58:57.076972 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:20.075309 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:59:20.075248 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:44.013221 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:59:44.013190 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffzjr_1ede441d-79f2-4365-8a08-403bf043ef73/ovn-acl-logging/0.log" Jul 06 18:59:44.013714 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:59:44.013496 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ffzjr_1ede441d-79f2-4365-8a08-403bf043ef73/ovn-acl-logging/0.log" Jul 06 18:59:50.252606 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:59:50.252532 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:50.375705 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:59:50.375661 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:53.079174 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:59:53.079139 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:53.186196 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:59:53.186163 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 18:59:57.544370 ip-10-0-128-98 kubenswrapper[2566]: I0706 18:59:57.544342 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:01.534914 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:00:01.534859 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:01.731496 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:00:01.731446 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:37.381346 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:00:37.381270 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 19:00:38.411185 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:00:38.411138 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:47.338358 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:01:47.338315 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:47.807776 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:01:47.807740 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:48.142036 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:01:48.141993 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:48.278679 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:01:48.278633 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:48.573194 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:01:48.573095 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:55.052672 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:01:55.052592 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:55.204065 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:01:55.204023 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:55.428650 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:01:55.428613 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm"] Jul 06 19:01:55.428855 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:01:55.428836 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" podUID="0b104471-f353-43b3-9beb-ee6c0d32dcd2" containerName="istio-proxy" containerID="cri-o://dbf6a3853e49072c183131f7888f597096932f738e75644b8eef5effb01a8ebd" gracePeriod=30 Jul 06 19:01:55.449297 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:01:55.449267 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:55.804225 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:01:55.804140 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:55.987815 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:01:55.987779 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:56.512392 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:01:56.512364 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:56.945768 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:01:56.945730 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:57.536711 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:01:57.536674 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 19:01:57.892489 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:01:57.892454 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 19:02:00.675515 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:00.675490 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 19:02:00.817484 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:00.817395 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istio-podinfo\") pod \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " Jul 06 19:02:00.817484 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:00.817437 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istio-token\") pod \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " Jul 06 19:02:00.817484 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:00.817459 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istio-envoy\") pod \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " Jul 06 19:02:00.817862 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:00.817497 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vlx67\" (UniqueName: \"kubernetes.io/projected/0b104471-f353-43b3-9beb-ee6c0d32dcd2-kube-api-access-vlx67\") pod \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " Jul 06 19:02:00.817862 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:00.817657 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-workload-socket\") pod \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " Jul 06 19:02:00.817862 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:00.817695 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-credential-socket\") pod \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " Jul 06 19:02:00.817862 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:00.817761 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istio-data\") pod \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " Jul 06 19:02:00.817862 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:00.817818 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-workload-certs\") pod \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " Jul 06 19:02:00.817862 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:00.817861 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istiod-ca-cert\") pod \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\" (UID: \"0b104471-f353-43b3-9beb-ee6c0d32dcd2\") " Jul 06 19:02:00.818235 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:00.817973 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0b104471-f353-43b3-9beb-ee6c0d32dcd2" (UID: "0b104471-f353-43b3-9beb-ee6c0d32dcd2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:02:00.818568 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:00.818361 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0b104471-f353-43b3-9beb-ee6c0d32dcd2" (UID: "0b104471-f353-43b3-9beb-ee6c0d32dcd2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:02:00.818568 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:00.818472 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-workload-socket\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 19:02:00.818568 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:00.818494 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-workload-certs\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 19:02:00.818817 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:00.818723 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0b104471-f353-43b3-9beb-ee6c0d32dcd2" (UID: "0b104471-f353-43b3-9beb-ee6c0d32dcd2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:02:00.818910 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:00.818824 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0b104471-f353-43b3-9beb-ee6c0d32dcd2" (UID: "0b104471-f353-43b3-9beb-ee6c0d32dcd2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:02:00.819040 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:00.819013 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istio-data" (OuterVolumeSpecName: "istio-data") pod "0b104471-f353-43b3-9beb-ee6c0d32dcd2" (UID: "0b104471-f353-43b3-9beb-ee6c0d32dcd2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:02:00.819944 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:00.819921 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0b104471-f353-43b3-9beb-ee6c0d32dcd2" (UID: "0b104471-f353-43b3-9beb-ee6c0d32dcd2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 19:02:00.820041 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:00.819972 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0b104471-f353-43b3-9beb-ee6c0d32dcd2" (UID: "0b104471-f353-43b3-9beb-ee6c0d32dcd2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:02:00.820041 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:00.819983 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b104471-f353-43b3-9beb-ee6c0d32dcd2-kube-api-access-vlx67" (OuterVolumeSpecName: "kube-api-access-vlx67") pod "0b104471-f353-43b3-9beb-ee6c0d32dcd2" (UID: "0b104471-f353-43b3-9beb-ee6c0d32dcd2"). InnerVolumeSpecName "kube-api-access-vlx67". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:02:00.820041 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:00.819995 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istio-token" (OuterVolumeSpecName: "istio-token") pod "0b104471-f353-43b3-9beb-ee6c0d32dcd2" (UID: "0b104471-f353-43b3-9beb-ee6c0d32dcd2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:02:00.919264 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:00.919220 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vlx67\" (UniqueName: \"kubernetes.io/projected/0b104471-f353-43b3-9beb-ee6c0d32dcd2-kube-api-access-vlx67\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 19:02:00.919264 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:00.919257 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-credential-socket\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 19:02:00.919264 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:00.919270 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istio-data\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 19:02:00.919519 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:00.919281 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istiod-ca-cert\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 19:02:00.919519 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:00.919293 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istio-podinfo\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 19:02:00.919519 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:00.919302 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istio-token\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 19:02:00.919519 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:00.919312 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b104471-f353-43b3-9beb-ee6c0d32dcd2-istio-envoy\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 19:02:01.404761 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:01.404729 2566 generic.go:356] "Generic (PLEG): container finished" podID="0b104471-f353-43b3-9beb-ee6c0d32dcd2" containerID="dbf6a3853e49072c183131f7888f597096932f738e75644b8eef5effb01a8ebd" exitCode=0 Jul 06 19:02:01.404975 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:01.404779 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" Jul 06 19:02:01.404975 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:01.404808 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" event={"ID":"0b104471-f353-43b3-9beb-ee6c0d32dcd2","Type":"ContainerDied","Data":"dbf6a3853e49072c183131f7888f597096932f738e75644b8eef5effb01a8ebd"} Jul 06 19:02:01.404975 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:01.404863 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm" event={"ID":"0b104471-f353-43b3-9beb-ee6c0d32dcd2","Type":"ContainerDied","Data":"5f5283bc96eea0a0eaaeb8121eab4b94eb7ed16e4d5d31fdfa20f41fb693615c"} Jul 06 19:02:01.404975 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:01.404905 2566 scope.go:117] "RemoveContainer" containerID="dbf6a3853e49072c183131f7888f597096932f738e75644b8eef5effb01a8ebd" Jul 06 19:02:01.415143 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:01.415124 2566 scope.go:117] "RemoveContainer" containerID="4b86e0428a469a1b0d044491c552486eeba82f02ad2343a526ee63ffd4f8714d" Jul 06 19:02:01.424006 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:01.423984 2566 scope.go:117] "RemoveContainer" containerID="dbf6a3853e49072c183131f7888f597096932f738e75644b8eef5effb01a8ebd" Jul 06 19:02:01.424293 ip-10-0-128-98 kubenswrapper[2566]: E0706 19:02:01.424268 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbf6a3853e49072c183131f7888f597096932f738e75644b8eef5effb01a8ebd\": container with ID starting with dbf6a3853e49072c183131f7888f597096932f738e75644b8eef5effb01a8ebd not found: ID does not exist" containerID="dbf6a3853e49072c183131f7888f597096932f738e75644b8eef5effb01a8ebd" Jul 06 19:02:01.424342 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:01.424305 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbf6a3853e49072c183131f7888f597096932f738e75644b8eef5effb01a8ebd"} err="failed to get container status \"dbf6a3853e49072c183131f7888f597096932f738e75644b8eef5effb01a8ebd\": rpc error: code = NotFound desc = could not find container \"dbf6a3853e49072c183131f7888f597096932f738e75644b8eef5effb01a8ebd\": container with ID starting with dbf6a3853e49072c183131f7888f597096932f738e75644b8eef5effb01a8ebd not found: ID does not exist" Jul 06 19:02:01.424342 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:01.424329 2566 scope.go:117] "RemoveContainer" containerID="4b86e0428a469a1b0d044491c552486eeba82f02ad2343a526ee63ffd4f8714d" Jul 06 19:02:01.424591 ip-10-0-128-98 kubenswrapper[2566]: E0706 19:02:01.424573 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b86e0428a469a1b0d044491c552486eeba82f02ad2343a526ee63ffd4f8714d\": container with ID starting with 4b86e0428a469a1b0d044491c552486eeba82f02ad2343a526ee63ffd4f8714d not found: ID does not exist" containerID="4b86e0428a469a1b0d044491c552486eeba82f02ad2343a526ee63ffd4f8714d" Jul 06 19:02:01.424638 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:01.424598 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b86e0428a469a1b0d044491c552486eeba82f02ad2343a526ee63ffd4f8714d"} err="failed to get container status \"4b86e0428a469a1b0d044491c552486eeba82f02ad2343a526ee63ffd4f8714d\": rpc error: code = NotFound desc = could not find container \"4b86e0428a469a1b0d044491c552486eeba82f02ad2343a526ee63ffd4f8714d\": container with ID starting with 4b86e0428a469a1b0d044491c552486eeba82f02ad2343a526ee63ffd4f8714d not found: ID does not exist" Jul 06 19:02:01.433117 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:01.433095 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm"] Jul 06 19:02:01.444964 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:01.444942 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-47f1eb-openshift-default-6555c447b7-dm9vm"] Jul 06 19:02:01.851932 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:01.851822 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b104471-f353-43b3-9beb-ee6c0d32dcd2" path="/var/lib/kubelet/pods/0b104471-f353-43b3-9beb-ee6c0d32dcd2/volumes" Jul 06 19:02:04.178275 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:04.178242 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 19:02:04.334346 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:04.334310 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 19:02:04.708831 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:04.708794 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6"] Jul 06 19:02:04.709039 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:04.709018 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" podUID="92a2600a-54f0-4228-8288-aee9e1b9da45" containerName="istio-proxy" containerID="cri-o://5542d250dbfc9858de0076edce9686c5b8f1c96ef22b9e3b4b638930e6b17f69" gracePeriod=30 Jul 06 19:02:04.737554 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:04.737519 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 19:02:04.970973 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:04.970883 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 19:02:05.359246 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:05.359148 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-56fc69f68d-lf7dv_48f39331-78fc-4928-b2d4-c17c1ed14760/authorino/0.log" Jul 06 19:02:09.959658 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:09.959630 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 19:02:10.107245 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.107153 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92a2600a-54f0-4228-8288-aee9e1b9da45-istio-podinfo\") pod \"92a2600a-54f0-4228-8288-aee9e1b9da45\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " Jul 06 19:02:10.107245 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.107204 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-istio-data\") pod \"92a2600a-54f0-4228-8288-aee9e1b9da45\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " Jul 06 19:02:10.107245 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.107227 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-istio-envoy\") pod \"92a2600a-54f0-4228-8288-aee9e1b9da45\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " Jul 06 19:02:10.107521 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.107257 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-workload-certs\") pod \"92a2600a-54f0-4228-8288-aee9e1b9da45\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " Jul 06 19:02:10.107521 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.107295 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-workload-socket\") pod \"92a2600a-54f0-4228-8288-aee9e1b9da45\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " Jul 06 19:02:10.107521 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.107326 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92a2600a-54f0-4228-8288-aee9e1b9da45-istio-token\") pod \"92a2600a-54f0-4228-8288-aee9e1b9da45\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " Jul 06 19:02:10.107521 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.107365 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-credential-socket\") pod \"92a2600a-54f0-4228-8288-aee9e1b9da45\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " Jul 06 19:02:10.107521 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.107413 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92a2600a-54f0-4228-8288-aee9e1b9da45-istiod-ca-cert\") pod \"92a2600a-54f0-4228-8288-aee9e1b9da45\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " Jul 06 19:02:10.107521 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.107498 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xd5xk\" (UniqueName: \"kubernetes.io/projected/92a2600a-54f0-4228-8288-aee9e1b9da45-kube-api-access-xd5xk\") pod \"92a2600a-54f0-4228-8288-aee9e1b9da45\" (UID: \"92a2600a-54f0-4228-8288-aee9e1b9da45\") " Jul 06 19:02:10.107823 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.107641 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "92a2600a-54f0-4228-8288-aee9e1b9da45" (UID: "92a2600a-54f0-4228-8288-aee9e1b9da45"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:02:10.107823 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.107659 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "92a2600a-54f0-4228-8288-aee9e1b9da45" (UID: "92a2600a-54f0-4228-8288-aee9e1b9da45"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:02:10.107823 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.107680 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "92a2600a-54f0-4228-8288-aee9e1b9da45" (UID: "92a2600a-54f0-4228-8288-aee9e1b9da45"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:02:10.108016 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.107914 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-workload-certs\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 19:02:10.108016 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.107936 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-workload-socket\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 19:02:10.108016 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.107949 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-credential-socket\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 19:02:10.108016 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.107990 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-istio-data" (OuterVolumeSpecName: "istio-data") pod "92a2600a-54f0-4228-8288-aee9e1b9da45" (UID: "92a2600a-54f0-4228-8288-aee9e1b9da45"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:02:10.108216 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.108071 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92a2600a-54f0-4228-8288-aee9e1b9da45-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "92a2600a-54f0-4228-8288-aee9e1b9da45" (UID: "92a2600a-54f0-4228-8288-aee9e1b9da45"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:02:10.109629 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.109603 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92a2600a-54f0-4228-8288-aee9e1b9da45-istio-token" (OuterVolumeSpecName: "istio-token") pod "92a2600a-54f0-4228-8288-aee9e1b9da45" (UID: "92a2600a-54f0-4228-8288-aee9e1b9da45"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:02:10.109743 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.109642 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/92a2600a-54f0-4228-8288-aee9e1b9da45-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "92a2600a-54f0-4228-8288-aee9e1b9da45" (UID: "92a2600a-54f0-4228-8288-aee9e1b9da45"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 19:02:10.109743 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.109705 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92a2600a-54f0-4228-8288-aee9e1b9da45-kube-api-access-xd5xk" (OuterVolumeSpecName: "kube-api-access-xd5xk") pod "92a2600a-54f0-4228-8288-aee9e1b9da45" (UID: "92a2600a-54f0-4228-8288-aee9e1b9da45"). InnerVolumeSpecName "kube-api-access-xd5xk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:02:10.109855 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.109801 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "92a2600a-54f0-4228-8288-aee9e1b9da45" (UID: "92a2600a-54f0-4228-8288-aee9e1b9da45"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:02:10.209321 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.209285 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xd5xk\" (UniqueName: \"kubernetes.io/projected/92a2600a-54f0-4228-8288-aee9e1b9da45-kube-api-access-xd5xk\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 19:02:10.209321 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.209314 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92a2600a-54f0-4228-8288-aee9e1b9da45-istio-podinfo\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 19:02:10.209321 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.209323 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-istio-data\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 19:02:10.209321 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.209331 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92a2600a-54f0-4228-8288-aee9e1b9da45-istio-envoy\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 19:02:10.209581 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.209338 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92a2600a-54f0-4228-8288-aee9e1b9da45-istio-token\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 19:02:10.209581 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.209346 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92a2600a-54f0-4228-8288-aee9e1b9da45-istiod-ca-cert\") on node \"ip-10-0-128-98.ec2.internal\" DevicePath \"\"" Jul 06 19:02:10.464538 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.464507 2566 generic.go:356] "Generic (PLEG): container finished" podID="92a2600a-54f0-4228-8288-aee9e1b9da45" containerID="5542d250dbfc9858de0076edce9686c5b8f1c96ef22b9e3b4b638930e6b17f69" exitCode=0 Jul 06 19:02:10.464713 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.464553 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" Jul 06 19:02:10.464713 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.464594 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" event={"ID":"92a2600a-54f0-4228-8288-aee9e1b9da45","Type":"ContainerDied","Data":"5542d250dbfc9858de0076edce9686c5b8f1c96ef22b9e3b4b638930e6b17f69"} Jul 06 19:02:10.464713 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.464649 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6" event={"ID":"92a2600a-54f0-4228-8288-aee9e1b9da45","Type":"ContainerDied","Data":"9c75185c63e153dc10ff6ffd3bf280bda20ac610c2fa9077ae5a0903cea5caea"} Jul 06 19:02:10.464713 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.464670 2566 scope.go:117] "RemoveContainer" containerID="5542d250dbfc9858de0076edce9686c5b8f1c96ef22b9e3b4b638930e6b17f69" Jul 06 19:02:10.474573 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.474558 2566 scope.go:117] "RemoveContainer" containerID="bb80e3f1913edee254ecce6deda38fe25d0d85c49d79fda1c329a2128ba0eb3b" Jul 06 19:02:10.483594 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.483575 2566 scope.go:117] "RemoveContainer" containerID="5542d250dbfc9858de0076edce9686c5b8f1c96ef22b9e3b4b638930e6b17f69" Jul 06 19:02:10.483848 ip-10-0-128-98 kubenswrapper[2566]: E0706 19:02:10.483826 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5542d250dbfc9858de0076edce9686c5b8f1c96ef22b9e3b4b638930e6b17f69\": container with ID starting with 5542d250dbfc9858de0076edce9686c5b8f1c96ef22b9e3b4b638930e6b17f69 not found: ID does not exist" containerID="5542d250dbfc9858de0076edce9686c5b8f1c96ef22b9e3b4b638930e6b17f69" Jul 06 19:02:10.483960 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.483860 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5542d250dbfc9858de0076edce9686c5b8f1c96ef22b9e3b4b638930e6b17f69"} err="failed to get container status \"5542d250dbfc9858de0076edce9686c5b8f1c96ef22b9e3b4b638930e6b17f69\": rpc error: code = NotFound desc = could not find container \"5542d250dbfc9858de0076edce9686c5b8f1c96ef22b9e3b4b638930e6b17f69\": container with ID starting with 5542d250dbfc9858de0076edce9686c5b8f1c96ef22b9e3b4b638930e6b17f69 not found: ID does not exist" Jul 06 19:02:10.483960 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.483902 2566 scope.go:117] "RemoveContainer" containerID="bb80e3f1913edee254ecce6deda38fe25d0d85c49d79fda1c329a2128ba0eb3b" Jul 06 19:02:10.484165 ip-10-0-128-98 kubenswrapper[2566]: E0706 19:02:10.484144 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb80e3f1913edee254ecce6deda38fe25d0d85c49d79fda1c329a2128ba0eb3b\": container with ID starting with bb80e3f1913edee254ecce6deda38fe25d0d85c49d79fda1c329a2128ba0eb3b not found: ID does not exist" containerID="bb80e3f1913edee254ecce6deda38fe25d0d85c49d79fda1c329a2128ba0eb3b" Jul 06 19:02:10.484206 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.484172 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb80e3f1913edee254ecce6deda38fe25d0d85c49d79fda1c329a2128ba0eb3b"} err="failed to get container status \"bb80e3f1913edee254ecce6deda38fe25d0d85c49d79fda1c329a2128ba0eb3b\": rpc error: code = NotFound desc = could not find container \"bb80e3f1913edee254ecce6deda38fe25d0d85c49d79fda1c329a2128ba0eb3b\": container with ID starting with bb80e3f1913edee254ecce6deda38fe25d0d85c49d79fda1c329a2128ba0eb3b not found: ID does not exist" Jul 06 19:02:10.503268 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.503233 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6"] Jul 06 19:02:10.516679 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:10.516646 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-b19850-openshift-default-768df9dc8c-nbkm6"] Jul 06 19:02:11.380473 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:11.380447 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722725-fgp2l_0da6f7d5-8441-418d-b90c-324d1e8649dd/cleanup/0.log" Jul 06 19:02:11.849811 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:11.849776 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92a2600a-54f0-4228-8288-aee9e1b9da45" path="/var/lib/kubelet/pods/92a2600a-54f0-4228-8288-aee9e1b9da45/volumes" Jul 06 19:02:12.355369 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:12.355338 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bf95b557-trqql_0b7b6e07-7bbd-4fe8-985e-d11852b486e8/copy-manifests/0.log" Jul 06 19:02:12.372438 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:12.372412 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bf95b557-trqql_0b7b6e07-7bbd-4fe8-985e-d11852b486e8/manager/0.log" Jul 06 19:02:12.488169 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:12.488142 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-cw9k8_5bd217af-1704-4e03-8ebe-8c300c47dbff/postgres/0.log" Jul 06 19:02:13.315594 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:13.315563 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4_30418921-3908-47df-8ea2-b5ff2737ba45/util/0.log" Jul 06 19:02:13.326000 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:13.325973 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4_30418921-3908-47df-8ea2-b5ff2737ba45/pull/0.log" Jul 06 19:02:13.347759 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:13.347739 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4_30418921-3908-47df-8ea2-b5ff2737ba45/extract/0.log" Jul 06 19:02:13.472004 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:13.471975 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs_d807c6c2-2ae7-4ed3-b810-eccf138dacc4/extract/0.log" Jul 06 19:02:13.491641 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:13.491616 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs_d807c6c2-2ae7-4ed3-b810-eccf138dacc4/util/0.log" Jul 06 19:02:13.505227 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:13.505202 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs_d807c6c2-2ae7-4ed3-b810-eccf138dacc4/pull/0.log" Jul 06 19:02:13.620002 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:13.619920 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q_423da052-df79-441b-9f40-235e7dd852cc/util/0.log" Jul 06 19:02:13.634063 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:13.634030 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q_423da052-df79-441b-9f40-235e7dd852cc/pull/0.log" Jul 06 19:02:13.645692 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:13.645665 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q_423da052-df79-441b-9f40-235e7dd852cc/extract/0.log" Jul 06 19:02:13.759199 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:13.759167 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg_e9a4973c-1b3d-4742-b736-0866b5383695/pull/0.log" Jul 06 19:02:13.768782 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:13.768756 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg_e9a4973c-1b3d-4742-b736-0866b5383695/extract/0.log" Jul 06 19:02:13.778645 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:13.778618 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg_e9a4973c-1b3d-4742-b736-0866b5383695/util/0.log" Jul 06 19:02:13.910551 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:13.910517 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-56fc69f68d-lf7dv_48f39331-78fc-4928-b2d4-c17c1ed14760/authorino/0.log" Jul 06 19:02:14.079363 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:14.079335 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-qmdl6_66bc78d7-1b98-4795-9097-641e96f71ba1/manager/0.log" Jul 06 19:02:14.455286 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:14.455252 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-4dc6g_e383c0fd-d33c-45bb-a05c-dbce41756fbf/registry-server/0.log" Jul 06 19:02:15.216892 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:15.216848 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4jvrqr_6e40633c-06a3-47fa-8c37-1401dc956580/istio-proxy/0.log" Jul 06 19:02:15.711833 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:15.711799 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-p5nb4_82159f17-c966-4747-aec4-1dd75e8726d5/istio-proxy/1.log" Jul 06 19:02:16.693317 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:16.693282 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-4nwkb_000ee753-6ecf-4054-9d24-a849f5d9c555/main/0.log" Jul 06 19:02:22.027850 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.027807 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cgk74/must-gather-pbb2z"] Jul 06 19:02:22.028392 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.028241 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 19:02:22.028392 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.028307 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b104471-f353-43b3-9beb-ee6c0d32dcd2" containerName="istio-proxy" Jul 06 19:02:22.028392 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.028322 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b104471-f353-43b3-9beb-ee6c0d32dcd2" containerName="istio-proxy" Jul 06 19:02:22.028392 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.028343 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92a2600a-54f0-4228-8288-aee9e1b9da45" containerName="istio-proxy" Jul 06 19:02:22.028392 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.028349 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="92a2600a-54f0-4228-8288-aee9e1b9da45" containerName="istio-proxy" Jul 06 19:02:22.028392 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.028365 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b104471-f353-43b3-9beb-ee6c0d32dcd2" containerName="istio-proxy" Jul 06 19:02:22.028392 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.028369 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b104471-f353-43b3-9beb-ee6c0d32dcd2" containerName="istio-proxy" Jul 06 19:02:22.028633 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.028435 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="92a2600a-54f0-4228-8288-aee9e1b9da45" containerName="istio-proxy" Jul 06 19:02:22.028633 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.028443 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="92a2600a-54f0-4228-8288-aee9e1b9da45" containerName="istio-proxy" Jul 06 19:02:22.028633 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.028449 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b104471-f353-43b3-9beb-ee6c0d32dcd2" containerName="istio-proxy" Jul 06 19:02:22.028633 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.028457 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="354c1db9-d4a3-42a0-84ac-fc5ef443b64e" containerName="istio-proxy" Jul 06 19:02:22.028633 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.028515 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92a2600a-54f0-4228-8288-aee9e1b9da45" containerName="istio-proxy" Jul 06 19:02:22.028633 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.028522 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="92a2600a-54f0-4228-8288-aee9e1b9da45" containerName="istio-proxy" Jul 06 19:02:22.028633 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.028594 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b104471-f353-43b3-9beb-ee6c0d32dcd2" containerName="istio-proxy" Jul 06 19:02:22.030834 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.030818 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cgk74/must-gather-pbb2z" Jul 06 19:02:22.036483 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.036458 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cgk74\"/\"kube-root-ca.crt\"" Jul 06 19:02:22.036914 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.036894 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cgk74\"/\"openshift-service-ca.crt\"" Jul 06 19:02:22.038083 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.038063 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-cgk74\"/\"default-dockercfg-tpbrd\"" Jul 06 19:02:22.056167 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.056138 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cgk74/must-gather-pbb2z"] Jul 06 19:02:22.126096 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.126056 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/16843a07-b3c8-4e05-acc7-a6e2337417c6-must-gather-output\") pod \"must-gather-pbb2z\" (UID: \"16843a07-b3c8-4e05-acc7-a6e2337417c6\") " pod="openshift-must-gather-cgk74/must-gather-pbb2z" Jul 06 19:02:22.126295 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.126107 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxvwt\" (UniqueName: \"kubernetes.io/projected/16843a07-b3c8-4e05-acc7-a6e2337417c6-kube-api-access-qxvwt\") pod \"must-gather-pbb2z\" (UID: \"16843a07-b3c8-4e05-acc7-a6e2337417c6\") " pod="openshift-must-gather-cgk74/must-gather-pbb2z" Jul 06 19:02:22.194103 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.194074 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" limit=1024 Jul 06 19:02:22.226566 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.226524 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/16843a07-b3c8-4e05-acc7-a6e2337417c6-must-gather-output\") pod \"must-gather-pbb2z\" (UID: \"16843a07-b3c8-4e05-acc7-a6e2337417c6\") " pod="openshift-must-gather-cgk74/must-gather-pbb2z" Jul 06 19:02:22.226755 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.226578 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxvwt\" (UniqueName: \"kubernetes.io/projected/16843a07-b3c8-4e05-acc7-a6e2337417c6-kube-api-access-qxvwt\") pod \"must-gather-pbb2z\" (UID: \"16843a07-b3c8-4e05-acc7-a6e2337417c6\") " pod="openshift-must-gather-cgk74/must-gather-pbb2z" Jul 06 19:02:22.226920 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.226898 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/16843a07-b3c8-4e05-acc7-a6e2337417c6-must-gather-output\") pod \"must-gather-pbb2z\" (UID: \"16843a07-b3c8-4e05-acc7-a6e2337417c6\") " pod="openshift-must-gather-cgk74/must-gather-pbb2z" Jul 06 19:02:22.252023 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.251993 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxvwt\" (UniqueName: \"kubernetes.io/projected/16843a07-b3c8-4e05-acc7-a6e2337417c6-kube-api-access-qxvwt\") pod \"must-gather-pbb2z\" (UID: \"16843a07-b3c8-4e05-acc7-a6e2337417c6\") " pod="openshift-must-gather-cgk74/must-gather-pbb2z" Jul 06 19:02:22.339832 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.339750 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cgk74/must-gather-pbb2z" Jul 06 19:02:22.487743 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.487715 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cgk74/must-gather-pbb2z"] Jul 06 19:02:22.488888 ip-10-0-128-98 kubenswrapper[2566]: W0706 19:02:22.488841 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16843a07_b3c8_4e05_acc7_a6e2337417c6.slice/crio-dc5fe699da029817e004eb708cc38d56f79b2c2fe9e3a765feb26497dcd443e8 WatchSource:0}: Error finding container dc5fe699da029817e004eb708cc38d56f79b2c2fe9e3a765feb26497dcd443e8: Status 404 returned error can't find the container with id dc5fe699da029817e004eb708cc38d56f79b2c2fe9e3a765feb26497dcd443e8 Jul 06 19:02:22.490952 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.490936 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:02:22.539143 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:22.539105 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cgk74/must-gather-pbb2z" event={"ID":"16843a07-b3c8-4e05-acc7-a6e2337417c6","Type":"ContainerStarted","Data":"dc5fe699da029817e004eb708cc38d56f79b2c2fe9e3a765feb26497dcd443e8"} Jul 06 19:02:23.547465 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:23.547431 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cgk74/must-gather-pbb2z" event={"ID":"16843a07-b3c8-4e05-acc7-a6e2337417c6","Type":"ContainerStarted","Data":"f03a55456a3191b29ffd8f1a1094d4b0bf11ec4c3fc7253f46d48ab5c580fdb6"} Jul 06 19:02:24.558398 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:24.558350 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cgk74/must-gather-pbb2z" event={"ID":"16843a07-b3c8-4e05-acc7-a6e2337417c6","Type":"ContainerStarted","Data":"f077721cd2fcc60ae697766b7d83c7542dc69b6857990ee39764dff345516fdb"} Jul 06 19:02:24.582765 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:24.582683 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-cgk74/must-gather-pbb2z" podStartSLOduration=2.717387952 podStartE2EDuration="3.58266009s" totalImagesPullingTime="865.272138ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:02:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:02:24.578848907 +0000 UTC m=+2561.305277867" watchObservedRunningTime="2026-07-06 19:02:24.58266009 +0000 UTC m=+2561.309089029" Jul 06 19:02:25.075144 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:25.075107 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5hbx6_3157c562-b33e-4c9c-b724-cdac9220bbb0/global-pull-secret-syncer/0.log" Jul 06 19:02:25.228598 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:25.228566 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-c2wvn_991513c4-afc5-4f9a-b436-648417e2b0e4/konnectivity-agent/0.log" Jul 06 19:02:25.379816 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:25.379738 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-98.ec2.internal_b7ccbef026f2cc3be335a3fabd6f8311/haproxy/0.log" Jul 06 19:02:29.847692 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:29.847571 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4_30418921-3908-47df-8ea2-b5ff2737ba45/extract/0.log" Jul 06 19:02:29.892266 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:29.892222 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4_30418921-3908-47df-8ea2-b5ff2737ba45/util/0.log" Jul 06 19:02:29.926091 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:29.926013 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fj4m4_30418921-3908-47df-8ea2-b5ff2737ba45/pull/0.log" Jul 06 19:02:29.977075 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:29.977050 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs_d807c6c2-2ae7-4ed3-b810-eccf138dacc4/extract/0.log" Jul 06 19:02:30.009431 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:30.009401 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs_d807c6c2-2ae7-4ed3-b810-eccf138dacc4/util/0.log" Jul 06 19:02:30.053884 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:30.053840 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05k7xs_d807c6c2-2ae7-4ed3-b810-eccf138dacc4/pull/0.log" Jul 06 19:02:30.090390 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:30.090363 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q_423da052-df79-441b-9f40-235e7dd852cc/extract/0.log" Jul 06 19:02:30.121077 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:30.120995 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q_423da052-df79-441b-9f40-235e7dd852cc/util/0.log" Jul 06 19:02:30.155005 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:30.154979 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rqt9q_423da052-df79-441b-9f40-235e7dd852cc/pull/0.log" Jul 06 19:02:30.197558 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:30.197526 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg_e9a4973c-1b3d-4742-b736-0866b5383695/extract/0.log" Jul 06 19:02:30.236178 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:30.236151 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg_e9a4973c-1b3d-4742-b736-0866b5383695/util/0.log" Jul 06 19:02:30.274767 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:30.274710 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1z5vwg_e9a4973c-1b3d-4742-b736-0866b5383695/pull/0.log" Jul 06 19:02:30.706181 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:30.706146 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-56fc69f68d-lf7dv_48f39331-78fc-4928-b2d4-c17c1ed14760/authorino/0.log" Jul 06 19:02:30.754226 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:30.753090 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-qmdl6_66bc78d7-1b98-4795-9097-641e96f71ba1/manager/0.log" Jul 06 19:02:30.863521 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:30.863490 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-4dc6g_e383c0fd-d33c-45bb-a05c-dbce41756fbf/registry-server/0.log" Jul 06 19:02:33.163611 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:33.163531 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_82733679-085c-4388-af3e-83eafbd94a81/alertmanager/0.log" Jul 06 19:02:33.204722 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:33.204652 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_82733679-085c-4388-af3e-83eafbd94a81/config-reloader/0.log" Jul 06 19:02:33.253574 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:33.253539 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_82733679-085c-4388-af3e-83eafbd94a81/kube-rbac-proxy-web/0.log" Jul 06 19:02:33.299892 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:33.295230 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_82733679-085c-4388-af3e-83eafbd94a81/kube-rbac-proxy/0.log" Jul 06 19:02:33.332927 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:33.332897 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_82733679-085c-4388-af3e-83eafbd94a81/kube-rbac-proxy-metric/0.log" Jul 06 19:02:33.367222 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:33.367192 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_82733679-085c-4388-af3e-83eafbd94a81/prom-label-proxy/0.log" Jul 06 19:02:33.403406 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:33.403375 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_82733679-085c-4388-af3e-83eafbd94a81/init-config-reloader/0.log" Jul 06 19:02:33.523101 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:33.523017 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-sxg68_ba103d37-a01d-4d1b-afab-5a92105dd1cc/kube-state-metrics/0.log" Jul 06 19:02:33.592204 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:33.592175 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-sxg68_ba103d37-a01d-4d1b-afab-5a92105dd1cc/kube-rbac-proxy-main/0.log" Jul 06 19:02:33.673899 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:33.673508 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-sxg68_ba103d37-a01d-4d1b-afab-5a92105dd1cc/kube-rbac-proxy-self/0.log" Jul 06 19:02:33.759393 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:33.759363 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6754684457-85h2d_db2ee9dc-cdec-4fd5-abc4-d049d53b829a/metrics-server/0.log" Jul 06 19:02:33.802091 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:33.802063 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-v67rj_57e592f6-5606-4945-b8e4-87fa68fa0977/monitoring-plugin/0.log" Jul 06 19:02:33.968772 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:33.968742 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k292x_63698879-bca0-4fb4-944b-3151be7fbc11/node-exporter/0.log" Jul 06 19:02:34.006087 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:34.006059 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k292x_63698879-bca0-4fb4-944b-3151be7fbc11/kube-rbac-proxy/0.log" Jul 06 19:02:34.036128 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:34.036100 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k292x_63698879-bca0-4fb4-944b-3151be7fbc11/init-textfile/0.log" Jul 06 19:02:34.181570 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:34.181542 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-qj66f_9bda0943-9d88-4367-be22-9ec970388f56/kube-rbac-proxy-main/0.log" Jul 06 19:02:34.219414 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:34.219385 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-qj66f_9bda0943-9d88-4367-be22-9ec970388f56/kube-rbac-proxy-self/0.log" Jul 06 19:02:34.250043 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:34.250000 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-qj66f_9bda0943-9d88-4367-be22-9ec970388f56/openshift-state-metrics/0.log" Jul 06 19:02:34.510783 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:34.510705 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-kn7k7_43bef29e-0780-4aaf-ad05-d8db529c0ece/prometheus-operator/0.log" Jul 06 19:02:34.533437 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:34.533393 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-kn7k7_43bef29e-0780-4aaf-ad05-d8db529c0ece/kube-rbac-proxy/0.log" Jul 06 19:02:34.561297 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:34.561267 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-jqslw_0d0bd084-df2e-4a5e-93ec-4142ba42228a/prometheus-operator-admission-webhook/0.log" Jul 06 19:02:34.675736 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:34.675707 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85bfd88756-s6985_f34f8e8d-3be3-4770-b3d8-9db846d5dcc6/thanos-query/0.log" Jul 06 19:02:34.701370 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:34.701342 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85bfd88756-s6985_f34f8e8d-3be3-4770-b3d8-9db846d5dcc6/kube-rbac-proxy-web/0.log" Jul 06 19:02:34.725356 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:34.725327 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85bfd88756-s6985_f34f8e8d-3be3-4770-b3d8-9db846d5dcc6/kube-rbac-proxy/0.log" Jul 06 19:02:34.749073 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:34.749044 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85bfd88756-s6985_f34f8e8d-3be3-4770-b3d8-9db846d5dcc6/prom-label-proxy/0.log" Jul 06 19:02:34.778963 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:34.778880 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85bfd88756-s6985_f34f8e8d-3be3-4770-b3d8-9db846d5dcc6/kube-rbac-proxy-rules/0.log" Jul 06 19:02:34.805594 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:34.805558 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85bfd88756-s6985_f34f8e8d-3be3-4770-b3d8-9db846d5dcc6/kube-rbac-proxy-metrics/0.log" Jul 06 19:02:35.229123 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:35.229083 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cgk74/perf-node-gather-daemonset-27zpb"] Jul 06 19:02:35.234915 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:35.234884 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-27zpb" Jul 06 19:02:35.243473 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:35.243444 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cgk74/perf-node-gather-daemonset-27zpb"] Jul 06 19:02:35.381246 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:35.381207 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e0949734-ba1b-48eb-a6cf-32f5c61e653d-lib-modules\") pod \"perf-node-gather-daemonset-27zpb\" (UID: \"e0949734-ba1b-48eb-a6cf-32f5c61e653d\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-27zpb" Jul 06 19:02:35.381428 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:35.381276 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e0949734-ba1b-48eb-a6cf-32f5c61e653d-podres\") pod \"perf-node-gather-daemonset-27zpb\" (UID: \"e0949734-ba1b-48eb-a6cf-32f5c61e653d\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-27zpb" Jul 06 19:02:35.381428 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:35.381341 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e0949734-ba1b-48eb-a6cf-32f5c61e653d-proc\") pod \"perf-node-gather-daemonset-27zpb\" (UID: \"e0949734-ba1b-48eb-a6cf-32f5c61e653d\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-27zpb" Jul 06 19:02:35.381428 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:35.381381 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxt26\" (UniqueName: \"kubernetes.io/projected/e0949734-ba1b-48eb-a6cf-32f5c61e653d-kube-api-access-qxt26\") pod \"perf-node-gather-daemonset-27zpb\" (UID: \"e0949734-ba1b-48eb-a6cf-32f5c61e653d\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-27zpb" Jul 06 19:02:35.381428 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:35.381414 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0949734-ba1b-48eb-a6cf-32f5c61e653d-sys\") pod \"perf-node-gather-daemonset-27zpb\" (UID: \"e0949734-ba1b-48eb-a6cf-32f5c61e653d\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-27zpb" Jul 06 19:02:35.482824 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:35.482738 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e0949734-ba1b-48eb-a6cf-32f5c61e653d-lib-modules\") pod \"perf-node-gather-daemonset-27zpb\" (UID: \"e0949734-ba1b-48eb-a6cf-32f5c61e653d\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-27zpb" Jul 06 19:02:35.482824 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:35.482820 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e0949734-ba1b-48eb-a6cf-32f5c61e653d-podres\") pod \"perf-node-gather-daemonset-27zpb\" (UID: \"e0949734-ba1b-48eb-a6cf-32f5c61e653d\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-27zpb" Jul 06 19:02:35.483085 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:35.482840 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e0949734-ba1b-48eb-a6cf-32f5c61e653d-proc\") pod \"perf-node-gather-daemonset-27zpb\" (UID: \"e0949734-ba1b-48eb-a6cf-32f5c61e653d\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-27zpb" Jul 06 19:02:35.483085 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:35.482863 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxt26\" (UniqueName: \"kubernetes.io/projected/e0949734-ba1b-48eb-a6cf-32f5c61e653d-kube-api-access-qxt26\") pod \"perf-node-gather-daemonset-27zpb\" (UID: \"e0949734-ba1b-48eb-a6cf-32f5c61e653d\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-27zpb" Jul 06 19:02:35.483085 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:35.482906 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0949734-ba1b-48eb-a6cf-32f5c61e653d-sys\") pod \"perf-node-gather-daemonset-27zpb\" (UID: \"e0949734-ba1b-48eb-a6cf-32f5c61e653d\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-27zpb" Jul 06 19:02:35.483085 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:35.482975 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e0949734-ba1b-48eb-a6cf-32f5c61e653d-lib-modules\") pod \"perf-node-gather-daemonset-27zpb\" (UID: \"e0949734-ba1b-48eb-a6cf-32f5c61e653d\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-27zpb" Jul 06 19:02:35.483085 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:35.482975 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e0949734-ba1b-48eb-a6cf-32f5c61e653d-podres\") pod \"perf-node-gather-daemonset-27zpb\" (UID: \"e0949734-ba1b-48eb-a6cf-32f5c61e653d\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-27zpb" Jul 06 19:02:35.483085 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:35.482985 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e0949734-ba1b-48eb-a6cf-32f5c61e653d-proc\") pod \"perf-node-gather-daemonset-27zpb\" (UID: \"e0949734-ba1b-48eb-a6cf-32f5c61e653d\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-27zpb" Jul 06 19:02:35.483085 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:35.483034 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0949734-ba1b-48eb-a6cf-32f5c61e653d-sys\") pod \"perf-node-gather-daemonset-27zpb\" (UID: \"e0949734-ba1b-48eb-a6cf-32f5c61e653d\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-27zpb" Jul 06 19:02:35.491756 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:35.491718 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxt26\" (UniqueName: \"kubernetes.io/projected/e0949734-ba1b-48eb-a6cf-32f5c61e653d-kube-api-access-qxt26\") pod \"perf-node-gather-daemonset-27zpb\" (UID: \"e0949734-ba1b-48eb-a6cf-32f5c61e653d\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-27zpb" Jul 06 19:02:35.555772 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:35.555734 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-27zpb" Jul 06 19:02:35.747145 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:35.746387 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cgk74/perf-node-gather-daemonset-27zpb"] Jul 06 19:02:35.750859 ip-10-0-128-98 kubenswrapper[2566]: W0706 19:02:35.750828 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode0949734_ba1b_48eb_a6cf_32f5c61e653d.slice/crio-afb8801c3aba8f479c1149827bc7d3b42e8dd7012016cb4e892bf64b66e094a1 WatchSource:0}: Error finding container afb8801c3aba8f479c1149827bc7d3b42e8dd7012016cb4e892bf64b66e094a1: Status 404 returned error can't find the container with id afb8801c3aba8f479c1149827bc7d3b42e8dd7012016cb4e892bf64b66e094a1 Jul 06 19:02:36.670094 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:36.670055 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-27zpb" event={"ID":"e0949734-ba1b-48eb-a6cf-32f5c61e653d","Type":"ContainerStarted","Data":"5d77334828a37fdf875b77adc6a72dd52688659877a5b2260ca91357bbdc25ec"} Jul 06 19:02:36.670597 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:36.670122 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-27zpb" event={"ID":"e0949734-ba1b-48eb-a6cf-32f5c61e653d","Type":"ContainerStarted","Data":"afb8801c3aba8f479c1149827bc7d3b42e8dd7012016cb4e892bf64b66e094a1"} Jul 06 19:02:36.670597 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:36.670325 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-27zpb" Jul 06 19:02:36.698762 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:36.698698 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-27zpb" podStartSLOduration=1.6986759070000002 podStartE2EDuration="1.698675907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:02:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:02:36.697423833 +0000 UTC m=+2573.423852804" watchObservedRunningTime="2026-07-06 19:02:36.698675907 +0000 UTC m=+2573.425104845" Jul 06 19:02:37.269513 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:37.269481 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5f6988b95b-7w4kf_8d33609e-12bd-4b5f-ba47-7bb64348eb3e/console/0.log" Jul 06 19:02:39.121007 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:39.120971 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rn4lz_8987aa45-6cac-4458-9d82-3a4e9c85e09e/dns/0.log" Jul 06 19:02:39.167994 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:39.167970 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rn4lz_8987aa45-6cac-4458-9d82-3a4e9c85e09e/kube-rbac-proxy/0.log" Jul 06 19:02:39.345469 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:39.345442 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vttwc_2a97d8c3-ea18-47d0-a315-59c693911c46/dns-node-resolver/0.log" Jul 06 19:02:40.003138 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:40.003110 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6cfff669f4-6fmtq_056b10b5-54ff-452a-94b2-24134a58f725/registry/0.log" Jul 06 19:02:40.135855 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:40.135818 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xczs7_27d5cb4d-462b-42ab-a879-fc2b7ae008b5/node-ca/0.log" Jul 06 19:02:41.204102 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:41.204060 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4jvrqr_6e40633c-06a3-47fa-8c37-1401dc956580/istio-proxy/0.log" Jul 06 19:02:41.490683 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:41.490595 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-p5nb4_82159f17-c966-4747-aec4-1dd75e8726d5/istio-proxy/0.log" Jul 06 19:02:41.527373 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:41.527335 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-p5nb4_82159f17-c966-4747-aec4-1dd75e8726d5/istio-proxy/1.log" Jul 06 19:02:42.396265 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:42.396237 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4chpk_01ea6ede-0121-412d-b17c-5d9073fbb347/serve-healthcheck-canary/0.log" Jul 06 19:02:42.688815 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:42.688727 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-27zpb" Jul 06 19:02:43.200605 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:43.200576 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-clcjg_e26e29f4-5957-4076-ab8b-3f741af97a3c/kube-rbac-proxy/0.log" Jul 06 19:02:43.238761 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:43.238731 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-clcjg_e26e29f4-5957-4076-ab8b-3f741af97a3c/exporter/0.log" Jul 06 19:02:43.272110 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:43.272075 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-clcjg_e26e29f4-5957-4076-ab8b-3f741af97a3c/extractor/0.log" Jul 06 19:02:46.308530 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:46.308500 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4jvrqr_6e40633c-06a3-47fa-8c37-1401dc956580/istio-proxy/0.log" Jul 06 19:02:46.567157 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:46.567126 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722725-fgp2l_0da6f7d5-8441-418d-b90c-324d1e8649dd/cleanup/0.log" Jul 06 19:02:46.595628 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:46.595600 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-p5nb4_82159f17-c966-4747-aec4-1dd75e8726d5/istio-proxy/0.log" Jul 06 19:02:46.641133 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:46.641087 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-p5nb4_82159f17-c966-4747-aec4-1dd75e8726d5/istio-proxy/1.log" Jul 06 19:02:47.263335 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:47.263243 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bf95b557-trqql_0b7b6e07-7bbd-4fe8-985e-d11852b486e8/manager/0.log" Jul 06 19:02:47.301296 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:47.301269 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bf95b557-trqql_0b7b6e07-7bbd-4fe8-985e-d11852b486e8/copy-manifests/0.log" Jul 06 19:02:47.357717 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:47.357679 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-cw9k8_5bd217af-1704-4e03-8ebe-8c300c47dbff/postgres/0.log" Jul 06 19:02:48.043807 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:48.043770 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4jvrqr_6e40633c-06a3-47fa-8c37-1401dc956580/istio-proxy/0.log" Jul 06 19:02:48.483622 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:48.483590 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-p5nb4_82159f17-c966-4747-aec4-1dd75e8726d5/istio-proxy/0.log" Jul 06 19:02:48.523385 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:48.523350 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-p5nb4_82159f17-c966-4747-aec4-1dd75e8726d5/istio-proxy/1.log" Jul 06 19:02:48.879613 ip-10-0-128-98 kubenswrapper[2566]: I0706 19:02:48.879515 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-77945c4cd8-7lw77_d1284b93-b89f-4487-b1c4-6cf7e61045f1/manager/0.log"